AFRL-HE-AZ-TR-2003-001 6 


UNITED  STATES  AIR  FORCE 
RESEARCH  LABORATORY 


FACILITATING  THE  ACQUISITION 
OF  MISSION  PLANNING 
AND  DYNAMIC  REPLANNING  EXPERTISE 


Kelly  Neville 
CHI  Systems,  Inc. 

12000  Research  Parkway,  Suite  120 
Orlando  FL  32826 

Jennifer  Fowlkes 

University  of  Central  Florida 
Institute  for  Simulation  and  Training 
3280  Progress  Drive 
Orlando  FL  32826 

Tamara  Strini 


CHI  Systems,  Inc. 

12000  Research  Parkway,  Suite  120 
Orlando  FL  32826 


JULY  2003 


Approved  for  public  release;  distribution  is  unlimited. 


AIR  FORCE  MATERIEL  COMMAND 
AIR  FORCE  RESEARCH  LABORATORY 
Human  Effectiveness  Directorate 
Warfighter  Training  Research  Division 
6030  South  Kent  Street 
Mesa  AZ  85212-6061 


NOTICES 


This  research  was  conducted  under  the  Small  Business  Innovation  Research  (SBIR) 
program  as  a  Phase  I  effort.  Publication  of  this  report  does  not  constitute  approval  or 
disapproval  of  the  ideas  or  findings.  It  is  published  in  the  interest  of  STINFO  exchange. 

The  US  Government  does  not  require  SBIR  Phase  I  contractors  to  adhere  to  any 
particular  format  or  style.  In  accordance  with  SBIR  guidelines  for  Phase  I  efforts,  the 
contractor’s  report  is  accepted  for  publication  but  is  not  edited. 

Using  Government  drawings,  specifications,  or  other  data  included  in  this 
document  for  any  purpose  other  than  Government-related  procurement  does  not  in  any 
way  obligate  the  US  Government.  The  fact  that  the  Government  formulated  or  supplied 
the  drawings,  specifications,  or  other  data,  does  not  license  the  holder  or  any  other 
person  or  corporation,  or  convey  any  rights  or  permission  to  manufacture,  use,  or  sell 
any  patented  invention  that  may  relate  to  them. 

The  Office  of  Public  Affairs  has  reviewed  this  report,  and  it  is  releasable  to  the 
National  Technical  Information  Service,  where  it  will  be  available  to  the  general  public, 
including  foreign  national. 

This  report  has  been  reviewed  and  is  approved  for  publication. 


ROBERT  T.NULLMEYER  HERBERT  H.  BELL 

Project  Scientist  Technical  Advisor 


CURTIS  J.  PAPKE,  Colonel,  USAF 

Chief,  Warfighter  Training  Research  Division 


Contract  Number:  F33615-02-M-6010 
Contractor:  CHI  Systems  Inc. 


Copies  of  this  reports  may  be  requested  from: 

Defense  Technical  Information  Center 
8725  John  J.  Kingman  Road,  Suite  0944 
Ft.  Belvoir,  VA  22060-6218 
http://stinet.dtic.mil 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing 
this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202- 
4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently 
valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


2.  REPORT  TYPE 


1.  REPORT  DATE  (DD-MM-YYYY) 
July  03 


4.  TITLE  AND  SUBTITLE 


Facilitating  the  Acquisition  of  Planning  and  Dynamic  Replanning  Expertise 


3.  DATES  COVERED  (From  -  To) 
February  2002  to  August  2002 


5a.  CONTRACT  NUMBER 

F33615-02-M-6010 


5b.  GRANT  NUMBER 


6.  AUTHOR(S) 

Kelly  Neville 
Jennifer  Fowlkes* 
Tamara  Strini 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 


CHI  Systems  Inc 
12000  Research  Parkway 
Suite  120 

Orlando  FL  32826 


*  University  of  Central  Florida 
Institute  for  Simulation  and  Training 
3280  Progress  Drive 
Orlando  FL  32826 


5c.  PROGRAM  ELEMENT  NUMBER 

65502F 


5d.  PROJECT  NUMBER 

3005 


5e.  TASK  NUMBER 

HA 


5f.  WORK  UNIT  NUMBER 

2H 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Research  Laboratory 
Human  Effectiveness  Directorate 
Warfighter  Training  Research  Division 
6030  South  Kent  Street 
Mesa  AZ  85212-6061 


12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  is  unlimited. 


10.  SPONSOR/MONITOR  S  ACRONYM(S) 

AFRL;  AFRL/HEA 


11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

AFRL-HE-AZ-T  R-2003-00 1 6 


13.  SUPPLEMENTARY  NOTES 

This  research  was  conducted  under  the  Small  Business  Innovation  Research  (SBIR)  program  as  a  Phase  I  effort. 

Air  Force  Research  Laboratory  Technical  Monitor:  Dr  Robert  T.  Nullmeyer,  AFRL/HEAS,  480.988.6561  x-283,  DSN  474-6283 


14.  ABSTRACT 

Two  current  trends-increasingly  complex  tactical  teams  and  the  growing  demand  for  dynamic  replanning-have  significant  implications  for 
mission  planning  and  the  ways  in  which  it  can  best  contribute  to  mission  success.  Effective  mission  planning  can  ficilitate  team 
coordination  both  during  the  relatively  predictable  phases  of  a  mission  and  during  more  challenging  mission  events  that  require  dynamic 
replanning  or  decision  making.  However,  it  is  hypothesized  that  aspects  of  mission  planning  expertise  that  facilitate  team  coordination 
during  dynamic  replanning  and  decision  making  are  not  exactly  the  same  as  those  that  support  team  coordination  during  more  predictable 
mission  phases.  On  the  basis  of  this  assumption,  we  proposed  to  develop  the  Cognition-Centered  Constructivistic  Program  of  Instruction 
(C3PI)  as  a  means  of  facilitating  the  acquisition  of  those  aspects  of  mission  planning  expertise  that  contribute  specifically  to  the  ability  of  a 
team  to  respond  to  dynamic,  on-the-fly  types  of  mission  events.  This  Phase  I  effort  involved  research  and  development  conducted  for  the 
purpose  of  developing  a  C3PI  system  design  that  is  grounded  in  theory  and  research  and  consistent  with  the  training  needs  and  constraints 
of  the  operational  user  community. 


15.  SUBJECT  TERMS 

C3PI;  Cognition  Centered  Constructivistic  Program  of  Instruction;  Decision  making;  Distributed  mission  training;  FY02  SBIR  Program; 
Phase  I  SBIR;  Planning;  Replanning;  Team  adaptability;  Team  coordination;  Simulation-based  training; 


19a.  NAME  OF  RESPONSIBLE  PERSON 

Ms  Liz  Casey 


19b.  TELEPHONE  NUMBER  (include  area 
code) 

480.988.6561  x-188  DSN  474-6188 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


16.  SECURITY  CLASSIFICATION  OF: 

a.  REPORT 

UNCLASSIFIED 

b.  ABSTRACT 

UNCLASSIFIED 

C.  THIS  PAGE 

UNCLASSIFIED 

17.  LIMITATION 

18.  NUMBER 

OF  ABSTRACT 

OF  PAGES 

UNLIMITED 

79 

PREFACE 


This  research  was  conducted  under  the  Small  Business  Innovation  Research 
(SBIR)  program  as  a  Phase  I  effort  by  CHI  Systems  Inc.  for  the  Air  Force  Research 
Laboratory,  Human  Effectiveness  Directorate,  Warfighter  Training  Research  Division 
(AFRL/HEA)  under  USAF  Contract  No.  F33615-02-M-6010,  and  Work  Unit  3005-HA- 
2H,  Cognitive  Demands  on  Warfighter  Readiness.  The  Laboratory  Contract  Monitor  was 
Dr  Robert  T.  Nullmeyer,  AFRL/HEAS. 

This  effort  benefited  from  the  support  and  involvement  of  a  number  of  people.  In 
particular,  the  effort  was  made  more  enjoyable  and  the  team  was  able  to  be  quite 
productive  thanks  to  active  involvement,  guidance,  and  support  provided  by  the  Air 
Force  sponsors,  Dr.  Robert  T.  Nullmeyer  (AFRL/HEAS)  and  Ms.  Anna  R.  Castillo 
(AFRL/HEAs).  In  addition,  we  are  particularly  indebted  to  Lt  Col  Michael  “Odie”  Park 
(AFRL/HEA)  for  generously  sharing  his  expertise  and  large  amounts  of  his  time. 
Without  his  assistance,  we  would  not  have  made  nearly  the  amount  of  progress  we 
achieved.  We  are  additionally  grateful  to  Joan  Ryder  of  CHI  Systems,  Inc.  for  her 
guidance  and  participation  during  this  effort. 


FACILITATING  THE  ACQUISITION  OF  MISSION  PLANNING 
AND  DYNAMIC  REPLANNING  EXPERTISE 


EXECUTIVE  SUMMARY 

Two  current  trends  -  increasingly  complex  tactical  teams  and  the  growing 
demand  for  dynamic  replanning  -  have  significant  implications  for  mission  planning  and 
the  ways  in  which  it  can  best  contribute  to  mission  success.  Effective  mission  planning 
can  facilitate  team  coordination  both  during  the  relatively  predictable  phases  of  a 
mission  and  during  more  challenging  mission  events  that  require  dynamic  replanning  or 
decision  making.  However,  it  is  hypothesized  that  the  aspects  of  mission  planning 
expertise  that  facilitate  team  coordination  during  dynamic  replanning  and  decision 
making  are  not  exactly  the  same  as  those  that  support  team  coordination  during  the 
more  predictable  mission  phases.  On  tie  basis  of  this  assumption,  we  proposed  to 
develop  the  Cognition-Centered  Constructivistic  Program  of  Instruction  (C3PI)  as  a 
means  of  facilitating  the  acquisition  of  those  aspects  of  mission  planning  expertise  that 
contribute  specifically  to  the  ability  of  a  team  to  respond  to  dynamic,  on-the-fly  types  of 
mission  events.  This  Phase  I  effort  involved  research  and  development  conducted  for 
the  purpose  of  developing  a  C3PI  system  design  that  is  grounded  in  theory  and 
research  and  consistent  with  the  training  needs  and  constraints  of  the  operational  user 
community. 

The  results  of  the  Phase  I  effort  are  described  in  this  report.  These  include  (a)  a 
summary  of  the  planning  and  replanning  expertise  literature;  (b)  a  summary  of  training 
strategies  identified  as  relevant  to  domains  such  as  air  combat  in  which  team 
adaptability  is  critical;  (c)  a  description  of  the  C3PI  conceptual  design;  (d)  a  description 
of  Phase  I  data  collection  and  analysis;  (e)  a  description  of  hypotheses  about  the 
cognitive  elements  of  expertise;  and  (f)  a  description  of  cognitive  task  analysis  (CTA)- 
based  research  plan  for  investigating  these  hypotheses  during  Phase  II. 

The  Phase  I  work  described  in  this  report  laid  the  groundwork  for  our  Phase  II 
effort,  which  will  involve  conducting  CTA-based  research  to  investigate  the  cognitive 
elements  of  planning  expertise,  refining  and  adding  to  the  C3PI  design  and  content 
based  on  the  findings  of  that  research,  implementing  the  design,  and  evaluating  the 
value  of  the  system,  both  in  terms  of  perceived  value  by  the  user  and  more  objective 
measures  of  performance  and  expertise  enhancement. 
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FACILITATING  THE  ACQUISITION  OF 
MISSION  PLANNING  AND  DYNAMIC  REPLANNING  EXPERTISE 

INTRODUCTION 

The  U.S.  Air  Force  Research  Laboratory's  Distributed  Mission  Training  (DMT) 
system  has  proven  its  effectiveness  in  providing  high-fidelity  training  to  a  variety  of 
military  teams.  However,  if  it  is  to  truly  support  the  creed  upon  which  it  was  founded  - 
that  warfighters  should  train  the  way  they  intend  to  fight  -  DMT  must  grow  in  certain 
important  ways.  One  of  these  ways  is  to  include  the  mission  planning  phase  in  DMT 
exercises.  Currently,  the  mission  plan  is  handed  to  crews  when  they  arrive  for  DMT 
exercises  and  mission  preparation  consists  mainly  of  preparing  and  delivering  a  mission 
brief  (R.  Nullmeyer,  personal  communication,  June  20,  2001).  Yet  the  notion  that  the 
quality  of  mission  planning  influences  whether  a  mission  is  executed  smoothly  is  deeply 
entrenched  in  military  philosophy.  Further,  the  ‘Top  Gun’  of  the  U.S.  Air  Force  -  the 
Fighter  Weapons  School  (FWS)  -  cites  mission  briefs,  thereby  implicating  mission 
planning,  as  one  of  the  primary  deficiencies  demonstrated  by  entering  pilots  (R. 
Nullmeyer,  personal  communication,  June  20,  2001). 

Although  the  mission  planning  process  and  plan  serve  numerous  important 
purposes,  one  of  the  potentially  most  important  purposes  is  to  prepare  teams  for  making 
and  executing  decisions  in  a  dynamic  environment.  Warfighters  operate  in  complex,  ill- 
structured,  and  time-pressured  environments,  requiring  adaptive  performance  for 
successful  mission  execution.  Adaptive  performance  underlies  success  in  many  military 
missions  and  it  can  be  argued  that  requirements  for  adaptability  are  increasing.  In 
particular,  quick  and  effective  responses  are  critical  in  today’s  threat  environment,  in 
which  the  threat  of  cold  war  has  been  replaced  by  multiple,  often  unpredictable, 
asymmetric  threats.  Further,  opponent  tactics  are  harder  to  anticipate  because  of  the 
lack  of  well-defined  templates  for  modern  adversaries  (Rosenberger,  1995). 
Technological  advances  in  military  systems  are  increasing  the  capabilities  as  well  as 
increasing  the  information  that  must  be  comprehended.  Increasingly,  too,  military 
teams  are  distributed,  joint,  and  multinational,  requiring  integration  of  platforms, 
terminology,  procedures,  and  tactics. 

Planning  can  contribute  to  adaptive  performance  during  mission  execution,  but 
neither  training  nor  guidance  is  rarely  provided  to  help  planners  better  prepare  for  this 
particularly  challenging  aspect  of  mission  execution.  Klein  and  Miller  (1999)  suggest 
that  planning  leads  to,  among  other  things,  the  generation  of  expectancies,  the 
identification  of  inconsistencies  and  planning  shortfalls  during  the  planning  process; 
better  improvisation  during  execution;  and  the  development  of  a  shared  understanding 
among  team  members,  allowing  them  to  direct  and  coordinate  their  actions.  Recent 
research  suggests  that  mission  performance  benefits  from  effective  planning.  Stout  and 
colleagues  (Stout,  1995;  Stout,  Cannon-Bowers,  Salas,  &  Milanovich,  1999)  found  that 
undergraduate  teams  judged  as  planning  well  used  more  efficient  communication 
strategies  under  high  workload  periods  and  exhibited  better  coordination  during  task 
performance.  Spiker,  Nullmeyer,  Tourville,  and  Silverman  (1997)  found  a  significant 
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correlation  between  planning  behavior  and  mission  execution  for  Special  Operation 
Forces  during  a  simulated  mission.  Mission  planning  behaviors  related  to  successful 
mission  execution  included  knowledge  of  threat  capabilities,  questioning  assumptions, 
and  extensive  “what-iffing.”  An  analysis  of  critical  incidents  obtained  through 
observations  of  Navy  air  wing  teams  during  distributed  team  training  found  that 
contingency  planning  and  “what  iffing”  were  crucial  performance  issues  (Bergondy, 
Fowlkes,  Gualtieri,  &  Salas,  1998). 

To  facilitate  development  of  such  planning  skills,  we  proposed  to  design  a 
planning  training  system  called  the  Cognition-Centered  Constructivistic  Program  of 
Instruction  (CP3I)  that  facilitates  the  acquisition  of  mission  planning  and  dynamic 
replanning  expertise  within  the  context  of  DMT  exercises.  DMT  offers  an  ideal 
environment  for  training  adaptive  performance  and,  similarly,  for  building  planning 
expertise  that  supports  adaptive  performance.  The  importance  of  distributed  simulation 
systems  such  as  DMT  as  a  means  to  train  adaptive  performance  was  noted  by 
Colegrove  and  Alliger  (2001 ,  p.  8): 

“The  ability  to  control  the  training  environment  provides  the 
opportunity  to  develop  individual  and  team  characteristics 
such  as  adaptability,  anticipation,  and  automaticity  to  a 
much  greater  extent  than  can  reliably  and  consistently  be 
attained  in  flight.  These  characteristics  lead  to  increased 
levels  of  performance  and  mission  success.” 

To  design  this  system  and  lay  the  groundwork  for  the  research  and  development 
we  will  need  to  accomplish  in  Phase  II,  we  performed  the  following  tasks  during  this 
Phase  I  effort: 

?  Conduct  of  a  literature  review  of  planning  and  replanning  expertise  and  training 

?  Review  of  training  strategies  relevant  to  air  combat  mission  planning  and  to  the 
air  combat  domain  in  general 

?  Development  of  a  preli  minary  design  concept  for  the  Cognition-Centered 
Constructivistic  Program  of  Instruction  (C3PI)  training  system 

?  Conduct  of  cognitive  task  analysis  (CTA)  interviews 

?  Development  of  hypotheses  about  the  cognitive  elements  of  planning  expertise 
that  contribute  most  to  dynamic  replanning  effectiveness 

?  Development  of  a  CTA  plan  for  studying  the  cognitive  elements  of  planning 
expertise  during  Phase  II 

In  the  sections  that  follow,  we  describe  the  results  of  these  tasks  in  turn  and  conclude 
with  a  discussion  of  the  research  and  development  to  be  conducted  during  Phase  II. 
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PHASE  I  RESULTS 


THE  ROLE  OF  PLANNING  IN  TEAM  ADAPTABILITY 

Studies  assessing  planning  and  replanning  strategies  in  tactical  teams  have 
identified  a  number  of  important  skills,  strategies,  and  behaviors.  In  our  review  of  the 
literature  on  planning  and  replanning  (also  referred  to  as  dynamic  and  on-the-fly 
decision  making),  we  were  particularly  interested  in  planning  skills,  strategies,  and 
behaviors  associated  with  effective  on-the-fly  replanning  and  decision  making  during 
mission  execution.  In  their  research  on  team  adaptability  in  army  units,  Klein  and 
Pierce  (2001 )  identified  a  number  of  team  skills  and  abilities  that  affect  a  team’s  ability 
to  react  effectively  to  unanticipated  challenges  encountered  during  mission  execution. 
These  ‘keys  to  success’  include: 

?  anticipating  problem  areas; 

?  adopting  a  proactive  problem  detection  mindset; 

?  preparing  for  modifying  a  planned  course  of  action  during  mission  execution; 

?  having  experience  with  solving  problems  as  a  team; 

?  having  the  big  picture  so  that  local/immediate  goals  don’t  overtake  the  team’s 
objectives; 

?  developing  a  repertoire  of  problem  solving  routines;  and 
?  having  a  good  sense  of  the  situation,  including  its  affordances  (i.e.,  features  that 
can  be  used  to  one’s  advantage). 

Although  it  can  be  argued  that  each  of  these  keys  to  success  may  be  achieved  or 
brought  about  through  effective  planning,  Klein  and  Pierce  do  not  directly  link  them  with 
planning.  Rather,  they  characterize  them  as  adaptation  skills  and  abilities,  asserting 
“We  see  team  adaptation  as  more  than  just  being  prepared...  (p.  7).”  In  fact,  these  skills 
and  abilities  may  represent  elements  of  expertise  that  both  directly  and  indirectly 
contribute  to  team  effectiveness  across  mission  phases  and  activities  (e.g.,  across 
planning,  replanning,  and  team  coordination  activities).  They  may  be  critical  to  effective 
planning  -  specifically,  an  effective  plan  may  be  one  in  which  mechanisms  are  built  in  to 
facilitate  these  skills  and  abilities  during  mission  execution.  Along  the  same  lines, 
planning  is  likely  critical  to  the  effective  use  of  these  skills  and  abilities  during  mission 
execution  -  that  is,  their  effective  use  would  depend  upon  the  foundation  of  a  solid 
mission  plan.  Mission  planning  additionally  represents  an  opportunity  to  acquire 
expertise  in  these  areas  of  skill  and  ability.  Specifically,  deliberate  planning  that 
involves  discussion  of  these  skills  and  abilities  and  how  to  develop  a  plan  that  best 
supports  them  can  be  used  as  an  opportunity  for  more  experienced  team  members  to 
help  less  experienced  members  focus  on  and  acquire  these  skills  and  abilities.  Hence, 
the  mission  preparation  process  is  both  critical  to  the  effective  use  of  these  skills  and 
abilities  and  relevant  to  their  acquisition. 

Klein  and  Pierce  (2001 )  additionally  suggest  types  of  training  that  can  be  used 
to  build  team  adaptability  and  avoid  team  ‘breakdown  points’.  As  in  the  case  of  the 
above  skills  and  abilities,  these  training  objectives  suggest  ways  mission  preparation 
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can  contribute  to  team  adaptability  -  both  as  a  training  opportunity  and  for  the  purpose 
of  developing  specific  types  of  information  in  advance.  These  training  objectives 
include: 

?  develop  a  large  repertoire  of  problem  solving  routines; 

?  develop  repertoire  of  self -organizing  mechanisms  (e.g.,  timelines  &  flow 
snapshots); 

?  train  for  anomalies,  workarounds,  and  replanning; 

?  develop  a  proactive  “adaptation  mindset”,  e.g.,  by  including  at  least  one 
malfunction  or  breakdown  in  each  training  exercise/scenario; 

?  calibrate  common  ground; 

?  train  for  both  external  and  internal  adaptation; 

?  train  to  manage  more  degrees  of  freedom; 

?  train  communication  workarounds; 

?  train  information  seeking  skills; 

?  train  to  rapidly  “parse”  a  task  (so  parts  of  task  can  be  reallocated);  and 

?  train  to  appreciate  the  team’s  affordances/resources  and  how  they  can  be 
combined  in  different  ways. 


Many  of  Klein  and  Pierce’s  recommendations,  although  based  on  work  with  army 
units,  are  relevant  to  other  types  of  tactical  teams,  including  air  combat  teams.  Other 
examples  of  critical  elements  of  expertise  that  contribute  to  team  adaptability  were 
identified  by  Klein  and  Miller  (1999),  who  suggest  that  adaptive  teams  generate 
expectancies  and  identify  inconsistencies  and  planning  shortfalls  during  the  planning 
process.  Other  researchers  have  examined  mission  planning  to  identify  planning  skills 
and  strategies  that  contribute  to  the  adaptability  of  tactical  teams  during  mission 
execution.  Identified  skills  and  strategies  include  developing  contingency  plans  (e.g., 
Bergondy,  Fowlkes,  Gualtieri,  &  Salas,  1998),  developing  thorough  contingency  plans 
(e.g.,  Macmillan,  Entin,  &  Serfaty,  1993),  identifying  problem  areas  (e.g.,  Crane,  1999), 
questioning  assumptions  (e.g.,  Bergondy  et  al.,  1 998;  MacMillan  et  al.,  1 993;  Spiker, 
Nullmeyer,  Tourville,  &  Silverman,  1997;  Spiker,  Nullmeyer,  &  Tourville,  2001), 
describing  and  rehearsing  situation-specific  radio  calls  (e.g.,  Nullmeyer,  Crane,  Cicero, 
&  Spiker,  2000),  and  using  a  what-iffing  strategy  to  evaluate  the  plan  (e.g.,  Bergondy  et 
al.,  1 998;  Crane,  1 999;  Spiker  et  al.,  1 997,  2001 ). 


Effective  planning  appears  to  facilitate  team  adaptability  by  allowing  team 
members  to  anticipate  the  information  needs  of  their  teammates  and  provide  that 
information  without  being  requested  to  do  so  (e.g.,  Orasanu,  1990;  Stout  et  al.,  1999). 
This  anticipatory  and  proactive  communication  strategy  is  especially  important  when 
communications  are  constrained,  as  in  large  and  complex  teams  (e.g.,  Gualtieri, 
Bergondy,  Oser,  &  Fowlkes,  1998)  and  in  high  workload  situations  (e.g.,  Orasanu,  1990; 
Stout,  et  al.,  1999). 
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Effective  planning,  both  prior  to  and  during  mission  execution  (i.e.,  dynamic 
replanning),  is  hypothesized  to  facilitate  team  adaptability  by  helping  team  members 
develop  shared  mental  models  or  common  ground  that  provide  them  with  an 
understanding  of  teammates’  expectations,  responsibilities,  and  informational 
requirements  (e.g.,  Klein  &  Miller,  1999;  Stout,  1995;  Stout,  Cannon-Bowers,  Salas,  & 
Milanovich,  1999).  Bergondy,  Fowlkes,  Gualtieri,  and  Salas  (1998)  assert  that,  during 
the  mission  brief,  it  is  critical  that  naval  air  wing  tactical  team  members  develop 
compatible  mental  models  of  the  mission,  and  that  in  the  execution  phase  it  is  critical  for 
them  to  update  these  models  and  use  them  to  identify  deviations  from  the  plan  and 
guide  interactions  with  other  team  members.  While  direct  evidence  is  lacking  (possibly 
due  to  the  difficulty  associated  with  measuring  mental  models),  this  hypothesis  is 
supported  by  research  showing  that  teamwork  and/or  interpositional  knowledge  training 
enhances  task  performance  (e.g.,  Cooke  et  al.,  2000).  Shared  mental  models  are 
thought  to  be  the  means  by  which  planning  helps  team  members  use  proactive 
communication  strategies  (e.g.,  Stout  &  Salas,  1993).  In  addition,  it  has  been 
suggested  that  shared  mental  models  allow  teams  to  use  implicit  coordination  strategies 
(e.g.,  Gualtieri,  et  al.,  1 998;  Entin  &  Serfaty,  1 999).  That  is,  team  members  are  thought 
to  act  and  make  decisions  based  on  these  shared  models  in  order  to  maintain 
performance  levels  when  high  workload  conditions  inhibit  overt  communication. 

Thus,  a  substantial  number  of  factors  related  to  successful  team  adaptability  in  a 
dynamic  environment  have  been  identified.  Importantly,  many  of  these  factors  are 
linked  to  the  mission  preparation  phase  and  planning,  in  particular.  These  include 
anticipation,  proactivity,  problem  detection,  preparation  for  change,  problem  solving 
strategies,  consideration  of  the  big  picture  and  long  term  goals,  situational  knowledge, 
strategies  for  building  common  ground,  and  planning  strategies  such  as  contingency 
planning,  what-iffing,  assumption  questioning,  identification  of  inconsistencies  and 
problem  areas,  and  rehearsal. 

TRAINING  STRATEGIES  THAT  ENHANCE  TEAM  ADAPTABILITY 

Recommendations  Based  on  Research  and  Theory 

The  emphasis  of  this  Phase  I  effort  was  not  just  on  how  teams  effectively  prepare 
for  dynamic  task  performance  environments,  but  also  on  how  to  best  train  teams  to 
effectively  prepare  for  them.  Accordingly,  we  reviewed  training  research  and  theory 
relevant  to  building  adaptability  in  teams.  Certain  training  strategies  and  guidelines 
described  in  this  literature  were  identified  as  particularly  relevant  to  our  training 
objectives  and  to  the  air  combat  mission  planning  domain.  These  relevant  strategies 
and  guidelines  are  summarized  here: 

?  Adaptability  is  enhanced  by  active  processing.  Adaptive  performance  requires 
an  active,  critically  thinking  human.  Klein,  Pliske,  Crandall,  and  Woods  (1999) 
performed  an  analysis  of  critical  incidents  across  domains  that  included 
firefighting,  US  Army  planners,  and  nurses  in  intensive  care  units.  Their  results 
suggest  that  problem  detection  is  an  active  process  and  that  operators  must  be 
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prepared  to  discover  anomalies.  This  principle  is  borne  out  of  a  number  of 
research  domains. 

-  Based  on  observations  of  Stability  and  Support  Operations,  Klein  and 
Pierce  (2001 )  argue  that  it  is  important  that  warfighters  develop  a 
proactive  mindset  and  be  prepared  to  recognize  situational  cue  patterns 
that  should  promote  adaptive  behavior. 

-  Pierce  and  Pomranky  (2001 ),  working  with  similar  military  teams, 
developed  an  Adaptive  Learning  Model  to  drive  a  training  system  concept. 
Guidance  from  this  model  suggests  the  importance  of  immersing  learners 
in  context-  relevant  simulations  and  exposing  them  to  realistic  problems. 

-  The  Adaptive  Learning  Model  is  based  in  part  on  the  theory  of  cognitive 
flexibility,  an  educational  approach  that  has  been  used  to  drive  advanced 
education  in  domains  such  as  medicine.  “Cognitive  flexibility,”  or  effective 
problem  solving  in  ill-structured  domains,  can  be  facilitated  by  advanced 
instructional  strategies  that  require  learners  to  use  the  same  knowledge 
elements,  but  from  different  perspectives,  prompted,  for  example,  by  the 
presentations  of  problems  and  case  studies  (Spiro,  Feltovich,  Jacobson,  & 
Coulson,  1992). 

-  Learner-generated  solutions  and  discovery  learning  lead  to  better  transfer 
to  novel  situations  (Smith,  Ford,  &  Kozlowski,  1997). 

-  Mental  simulation  is  another  training  strategy  that  encourages  active 
processing.  Cohen,  Thompson,  Adelman,  Bresnick,  Tolcott  &  Freeman 
(1995)  stress  critical  thinking  and  deliberate  mental  simulation  in  the 
recognition-meta-recognition  model.  According  to  this  model,  once  a 
situation  is  recognized,  it  can  be  subjected  to  a  number  of  tests,  similar  to 
the  way  good  readers  develop  and  actively  test  hypotheses  about  their 
comprehension.  Kozlowski  (1998)  recommends  mental  rehearsal  as  a 
training  intervention  that  encourages  trainees  to  think  through 
contingencies. 

Adaptability  should  be  trained  in  context.  Research  suggests  that  the 
development  of  adaptive  performance  requires  “domain-specific”  practice  (Pierce 
&  Pomranky,  2001).  Kozlowski  (1998,  p.1 1 6)  noted: 

“Whereas  basic  skills  can  be  developed  in  conventional 
training  environments  (i.e.,  the  classroom),  adaptive  skills  are 
fully  developed  and  refined  in  the  performance  environment. 

This  means  shifting  more  training  to  the  performance  context 
and  developing  new  training  strategies  and  techniques  that  can 
be  integrated  into  the  work  environment.” 

Adaptability  is  enhanced  by  the  development  of  links  between  key  knowledge 
elements  in  the  domain.  Training  strategies  to  improve  adaptive  performance 
provide  context  specific  learning  prompts  to  understand  the  complex 
interrelationships  among  domain  features  (Kozlowski,  1998;  Pierce  &  Pomranky, 
2001 ;  Spiro  et  al.,  1992). 


?  Team  adaptability  is  supported  by  shared  mental  models.  Researche  rs  have 
argued  that  team  adaptability  is  supported  by  team  members  who  have 
consistent  expectations  for  how  environmental  factors  should  guide  actions.  For 
example,  in  military  teams,  it  is  crucial  for  teams  to  have  a  common 
understanding  of  how  factors  such  as  commander’s  intent  and  the  enemy  order 
of  battle  should  influence  decisions  and  actions. 

Research  on  both  planning  and  training  for  dynamic  environments  is  consistent 
with  the  use  of  a  what-iffing  type  of  strategy  to  build  team  adaptability.  In  addition  to 
being  associated  with  effective  planning  (as  described  in  the  preceding  section),  this 
type  of  strategy  is  recommended  by  Klein  and  Pierce  (2001)  as  a  means  for  building 
adaptive  teams  and  is  represented  in  the  Adaptive  Battlefield  Thinking  program  in  the 
form  of  disequilibrium  experiences  (i.e.,  events  of  a  difficulty  level  with  which  trainees 
have  not  yet  gained  experience  and  which  causes  them  to  question  and  re-evaluate 
their  assumptions  and  strategies;  Ross  &  Pierce,  2000).  Further,  the  strategy  is  already 
used  extensively  in  training  by  various  operational  military  communities,  as  discussed 
below. 

Lessons  from  Military  Operational  Communities 

Although  planning  for  uncertain  environments  and  dynamic  replanning  within 
those  environments  has  received  heightened  attention  in  the  past  decade  (e.g.,  due  to 
the  high  priority  of  time  critical  targets),  they  have  been  pervasive  aspects  of  tactical 
mission  performance  throughout  history.  Accordingly,  the  military  services  and 
branches  have  evolved  planning  and  training  techniques  that,  although  not  officially 
recognized  for  it,  prepare  personnel  for  planning  and  replanning  in  dynamic 
environments.  These  techniques  include  strategies  that  help  build  team  common 
ground  and  shared  mental  models  (e.g.,  the  shared  use  of  planning  maps  and  planning 
whiteboards,  and  sand  tables  around  which  multiple  team  elements  can  gather)  and 
strategies  that  help  foster  proactive  thinking,  assumption  questioning,  and  contingency 
preparation  during  planning. 

This  latter  type  of  strategy  typically  takes  the  form  of  what-iffing.  For  example,  in 
U.S  Marine  Corp  Combined  Arms  Staff  Trainer  (CAST)  exercises,  expert  CAST 
exercise  controllers  look  for  weaknesses  in  trainee  plans  and  then  insert  what-if 
challenges  into  sand  table  exercises  to  target  those  weaknesses.  As  another  example, 
during  integrated  naval  air  wing  training  held  at  the  Naval  Strike  and  Air  Warfare  Center 
(NSAWC),  instructors  sit  in  on  briefing  sessions  in  order  to  identify  weaknesses  in  the 
briefed  plan.  Subsequently,  they  may  introduce  challenges  into  the  training  mission 
(e.g.,  a  pop-up  surface-to-air  missile  [SAM]  or  a  threat  in  the  target  area)  that  are 
designed  to  exploit  those  weaknesses.  Thus,  they  what-if  the  plan  during  the  execution 
of  the  mission  -  an  effective  way  of  demonstrating  to  trainees  the  importance  of 
proactive  thinking,  assumption  questioning,  and  contingency  planning  during  the 
planning  phase.  Air  force  fighter  pilots  use  a  what-iffing  strategy,  as  well.  Similar  to  the 
naval  air  wing  example,  one  of  the  fighter  pilots  designated  to  play  the  adversary  role 
during  a  given  training  mission  may  listen  to  the  training  team’s  mission  brief  of  the  in 
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order  to  identify  the  weaknesses  in  their  plan.  The  adversarial  pilot  then  joins  with  the 
rest  of  the  adversarial  team  to  develop  a  plan  that  will  exploit  those  weaknesses.  Again, 
the  technique  features  what-iffing  during  mission  execution  instead  of  during  planning, 
but  it  nonetheless  is  an  effective  way  of  driving  home  the  benefit  to  be  gained  by  what- 
iffing  prior  to  mission  execution. 

In  these  examples,  what-iffing  takes  place  during  the  mission  execution  or,  in  the 
case  of  CAST  exercises,  during  the  simulation  phase  of  training  exercises  rather  than 
during  the  mission  preparation  phase.  This  approach  treats  mission  execution  and 
simulation  that  takes  place  in  the  context  of  training  as  a  form  of  real  world  mission 
preparation.  That  is,  rather  than  being  limited  to  what-iffing  in  a  planning  room  using 
only  a  whiteboard,  markers,  and  the  team’s  cognitive  resources,  what-iffing  is 
conducted  in  the  hands-on  environment  of  a  high-fidelity  simulation  or  live  training 
mission.  The  benefits  of  this  approach  include  the  opportunity  to  practice  responding  to 
dynamic  replanning  and  decision  events  under  more  realistic  conditions.  The  approach 
additionally  can  benefit  mission  preparation  and  possibly  improve  the  use  of  what-iffing 
during  the  planning  phase  of  real  world  missions.  Specifically,  the  insertion  of  what-if 
events  during  mission  execution  or  simulation  serves  to  make  trainees  aware  of  both 
the  weaknesses  in  their  plan  and  (sometimes  painfully  aware)  of  the  consequences  of 
those  weaknesses.  In  addition,  what-if  training  strategies  enrich  trainees’  knowledge 
base  through  exposure  to  realistic  what-if  experiences.  These  what-if  events  thus 
become  sources  of  experience  that  a  trainee  may  subsequently  draw  upon  to  what-if 
during  real-world  missions. 

What-iffing  is  also  used  in  air  combat  mission  planning  rooms,  although  the 
extent  to  which  it  is  used  and  its  quality  varies  considerably  depending  on,  for  example, 
the  team  lead,  mission  type,  and  planning  constraints.  Thus,  what-iffing  is  a  technique 
that  is  already  viewed  as  effective  by  combat  mission  planning  teams.  Because  of  its 
acceptance  and  because  research  suggests  that  it  plays  an  important  role  in  team 
adaptability  and  mission  preparedness,  what-iffing  was  adopted  as  the  training  strategy 
around  which  the  C3PI  system  was  developed.  C3PI  will  encourage  what-iffing  during 
mission  planning  both  via  its  training  mechanisms  and  by  making  the  process  more 
stimulating  through  the  provision  of  a  variety  of  events,  challenges,  and  tools.  It  will 
additionally  support  trainees,  instructors,  and  planners  by  making  available  a  wide 
range  of  challenging  what-if  possibilities.  Thus,  C3PI  is  designed  to  provide  some  of  the 
same  training  benefits  associated  with  what-iffing  during  mission  execution.  Although 
this  training  will  not  necessarily  make  trainees  aware  of  the  potential  consequences  of 
insufficient  planning  in  the  way  the  techniques  described  above  do  (as  a  marine  has 
described  it,  “they  will  learn  if  they  bleed”),  it  will  include  a  number  of  tools  that  help  the 
planner  think  through  the  plan  elements  in  a  comprehensive  way  and  consider  various 
potential  consequences.  Furthermore,  it  will  expose  trainees  to  a  potentially  large 
number  of  what-if  events  and  what-if  challenges  associated  with  different  levels  of 
difficulty,  thereby  building  their  ‘what-if  knowledge  base. 

C3PI  offers  training  benefits  in  addition  to  those  provided  by  what-iffing.  These 
additional  benefits  stem  from  the  fact  that  although  C3PI  is  designed  to  support  the 
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conduct  of  what-iffing  exercises,  the  exercises  are,  in  fact,  used  as  a  ‘training  delivery 
vehicle  or  mechanism.  What-iffing  is  a  valuable  experientially-based  training  technique 
that  is  accepted  by  the  operational  community.  As  such,  what-iffing  exercises  represent 
an  effective  means  for  implementing  and  integrating  a  number  of  other  training 
techniques  aimed  at  building  expertise  in  planning. 

Although  the  C3PI  tools  and  features  are  described  in  detail  below,  a  few  are 
briefly  introduced  here  to  demonstrate  how  the  what-iffing  paradigm  can  be  used  to 
integrate  multiple  training  strategies  and  objectives.  For  example,  the  C3PI  design 
specifies  planning  tools  that  give  trainees  access  to  lessons  learned  and  video  clips  of 
experts  relating  advice.  These  tools  represent  a  form  of  analogy-  or  case-based 
learning,  as  noted  previously.  The  Platform  Specs  tool  represents  a  form  of  cross 
training  and  a  means  of  building  critical  types  of  knowledge  -  knowledge  about  team 
members  and  knowledge  about  enemy  capabilities.  Another  training  objective  of  C3PI 
is  to  help  trainees  build  the  knowledge  and  knowledge  structures  that  support  effective 
what-if  planning.  C3PI  is  designed  to  meet  this  objective  using  a  scaffolding,  event- 
based  training  approach  and  a  form  of  attentional  advice  (i.e. ,  knowledge-use  prompts; 
e.g.,  Cannon-Bowers,  Rhodenizer,  Salas,  &  Bowers,  1998).  It  is  anticipated  that 
proposed  Phase  II  research  will  reveal  additional  aspects  of  planning  expertise  and  that 
training  strategies  designed  to  target  those  aspects  will  be  integrated  into  the  C3PI 
what-iffing  framework. 

C3PI  additionally  will  make  evident  to  trainees  the  linkages  between  mission 
planning  and  execution,  and  it  will  do  this  using  an  event-based  training  type  of 
approach  (e.g.,  Fowlkes  et  al.,  1 998).  Links  will  be  emphasized  by  exposing  DMT 
teams  to  what-if  events  during  DMT  mission  execution,  including  some  events  that  were 
previously  what-iffed  using  C3PI  and  others  that  were  not,  and  then  by  asking  the  teams 
to  discuss  during  the  After  Action  Review  (AAR)  the  ways  in  which  the  what-iffing  they 
performed  prior  to  mission  execution  was  helpful  and  in  what  ways  it  could  have  been 
better.  As  part  of  the  C3PI  Phase  II  effort,  we  will  perform  research  to,  among  other 
things,  identify  performance  measures  that  are  affected  by  what-iffing  effectiveness 
during  mission  preparation.  These  measures  will  also  be  used  to  demonstrate  this  link 
to  trainees. 

DEVELOPMENT  OF  A  PRELIMINARY  CONCEPT  DESIGN 

In  this  section,  we  describe  the  Phase  I  conceptual  C3PI  design.  First  we  give 
an  overview  of  the  rationale  for  the  what-iffing  framework  used  as  the  core  of  the 
design.  Then  the  conceptual  design  is  described  -  first,  in  terms  of  a  concept  of 
operations  for  how  a  user  would  interact  with  the  system  and  then  a  description  of 
systems  specifications  and  requirements.  This  section  is  followed  by  a  section  that 
describes  research  conducted  in  association  with  this  effort,  and  a  description  of  future 
work  to  conclude  the  report. 
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Influences 


Increasingly  complex  tactical  teams  and  the  growing  demand  for  dynamic 
replanning  -  have  significant  implications  for  mission  planning  and  the  ways  in  which  it 
can  best  contribute  to  mission  success.  Effective  mission  planning  can  facilitate  team 
coordination  both  during  the  relatively  predictable  phases  of  a  mission  and  during  more 
challenging  mission  events  that  require  dynamic  replanning  or  decision  making. 
However,  it  is  hypothesized  that  the  aspects  of  mission  planning  expertise  that  facilitate 
team  coordination  during  dynamic  replanning  and  decision  making  are  not  exactly  the 
same  as  those  that  support  team  coordination  during  the  more  predictable  mission 
phases.  On  the  basis  of  this  assumption,  we  proposed  to  develop  C3PI  as  a  means  of 
facilitating  the  acquisition  of  those  aspects  of  mission  planning  expertise  that  contribute 
specifically  to  the  ability  of  a  team  to  respond  to  dynamic,  on-the-fly  types  of  mission 
events. 

The  C3PI  conceptual  design  represents  a  planning  and  training  system  that  is 
based  on  what-if  exercises.  As  discussed  above,  what-iffing  is  an  experientially-based 
training  technique  that  is  accepted  by  the  operational  community  and  has  been  found  by 
researchers  to  contribute  to  mission  success  and  team  adaptability.  Its  use  is 
consistent  with  the  training  guidelines  described  above,  as  what-iffing  enhances  active 
processing  and  critical  thinking  skills  and  provides  context -specific  opportunities  for 
mental  simulation  and  for  building  relevant  domain  knowledge.  Further,  what-iffing 
serves  as  a  rich  context  for  introducing  a  number  of  other  training  techniques  aimed  at 
building  expertise  in  planning.  Thus,  what-iffing  additionally  is  used  within  C3PI  as  a 
vehicle  for  applying  multiple  complementary  training  strategies  in  an  integrated  way  to 
help  trainees  acquire  expert  planning  skills,  strategies,  and  knowledge  to  improve  on- 
the-fly  decision  making  and  dynamic  replanning. 

C3PI  is  based  on  a  training  philosophy  similar  to  event-based  training  in  which 
the  mission  briefing,  execution,  and  debriefing  phases  represent  opportunities  to  extend 
the  training  benefits  associated  with  using  C3PI  during  the  planning  phase  (e.g., 
Fowlkes,  Dwyer,  Oser,  &  Salas,  1998).  Accordingly,  C3PI  will  integrate  with  briefing, 
performance  assessment,  and  AAR  technologies  and  phases.  The  C3PI  application 
initially  will  be  developed  to  support  DMT  training  exercises  (although  it  may  also  be 
used  independently  of  them)  and  is  being  developed  for  collaborative  use  across 
distributed  sites. 

C3PI  Concept  of  Operations 

During  Phase  I,  a  conceptual  C3PI  design  was  developed  based  on  research 
literature,  subject-matter  expert  (SME)  interviews,  and  procedures  followed  by  the 
operational  community,  as  described  above.  In  this  section,  we  describe  a  C3PI 
training  session  in  terms  of  how  a  new  user  would  interact  with  the  system.  Figure  1 
depicts  a  sequence  of  high-level  interactions  associated  with  this  training  session.  The 
description  below  assumes  that  the  user  participated  in  the  initial  baseline  DMT 
scenario  and  now  wants  to  use  C3PI  as  part  of  preparing  for  the  first  training  scenario. 
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Select 

Example  scenario  is 

mission 

‘4  x  X  DCA  Point  Defense  vs. 

scenario 

Short  Burn  Threat’ 

T 

3-min  video  clip  and 
graphical  presentation 
of  scenario 


4  ^ 


Did  you 

Review  a 

develop  plan 

ready- 

yet? 

made  plan 

(yesj^  $ 


Select  event  from  list 
of  what-if  events 

a 

Select  challenge  from 
list  of  what-if  challenges 

a 

Develop  plan  using  C3PI 
planning  environment 


a 


Access  what-if  plans  (snapshots 
and  supporting  information)  during 
brief  and  AAR;  Plans  can  also  be 
printed  and  carried  into  aircraft 


User  sees  SPINS,  map  of  threat 
laydown  and  boundaries,  and 
video  of  intel  brief  followed  by  pilot 
giving  brief  synopsis  of  mission 


User  should  develop  a  basic  mission 
plan  or  review  a  ready-made  plan 
prior  to  what-iffing;  Ready-made  plan 
option  may  be  selectively  blocked  for 
DMT  participants 


Example  events  are  ‘3-Group  Wall’ 
and  ‘1  Heavy  Group  and  1  Light 
Group  in  Azimuth’ 


Within  each  what-if  event,  what-if 
challenges  are  posed  iteratively 


User  develops  annotated  snapshot 
of  plan  and  fills  in  supporting 
information  panel;  may  use  planning 
support  and  training  tools 


Plans  can  be  changed  during 
briefing  and  AAR  sessions;  lessons 
learned  can  be  added  to  team 
profile;  Debrief  agent  supports  team 
self-correction  during  AAR 


Figure  1.  User  interactions  with  C3PI  during  what-if  planning  exercises. 
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Entering  a  user  profile.  After  the  user  initiates  the  C3PI  session,  he  will  be 
asked  to  create  a  user  account  and  to  provide  information  that  will  become  part  of  his 
user  profile.  He  will  be  prompted  to  identify  his  DMT  team  so  that  he  can  access  shared 
team-level  information  including  DMT  team  performance  metrics  and  lessons  learned 
accumulated  by  the  team  across  the  week  of  exercises.  In  addition,  he  will  be  prompted 
to  enter  training  and  experience  data  so  the  system  can  approximate  his  level  of 
expertise.  C3PI  will  automatically  track  each  user’s  progress  within  C3PI  with  respect  to 
syllabi  and  scenarios  practiced  and  with  respect  to  experience  with  different  event  types 
and  knowledge  categories. 

Selecting  a  syllabus  and  scenario.  After  initiating  an  account  and  entering  user 
profile  data  (or  after  logging  on  if  it  is  not  the  user’s  first  session),  the  user  will  be 
prompted  to  choose  a  DMT  syllabus  (he  will  be  asked  to  choose  the  one  he  is  following 
during  that  particular  week-long  exercise)  and  then  a  scenario  from  the  syllabus.  The 
user  will  be  encouraged  (e.g.,  using  automated  feedback)  to  choose  the  first  scenario 
that  has  not  previously  been  chosen  (scenarios  are  ordered  according  to  approximate 
difficulty).  After  the  scenario  has  been  chosen,  a  scenario  or  mission  synopsis  is 
presented  to  the  user.  This  synopsis  consists  of  a  text  presentation  of  the  basic  mission 
parameters,  including  special  instructions  (SPINS),  a  graphical  depiction  of  boundaries 
and  the  enemy  order  of  battle,  and  a  brief  video  clip  presentation  featuring  an  intel 
officer  and  then  a  pilot  briefing  the  threat  situation  and  the  scenario  objectives, 
respectively. 

In  case  the  user  has  not  already  developed  a  basic  mission  plan  for  this 
particular  scenario,  C3PI  will  prompt  them  to  do  so  following  the  scenario  synopsis. 
Alternatively,  if  the  user  is  only  using  the  system  to  practice  what-iffing,  he  will  be  able 
to  request  a  brief  of  a  ready-made  mission  plan  for  the  scenario,  which  will  be  presented 
via  a  combination  of  video  clip  and  text-based  briefing  charts.  The  ready-made  plan 
option  may  be  selectively  blocked  so  that  DMT  participants  will  be  forced  to  develop 
their  own  mission  plan. 

Selecting  what-if  events.  After  the  scenario  synopsis  is  presented,  the  user  is 
given  access  to  a  list  of  what-if  events.  What-if  events  will  be  situations  that  might  occur 
within  a  scenario.  They  will  largely  consist  of  various  threat  presentations  but  will 
additionally  include  other  types  of  challenges  that  require  on-the-fly  replanning  or 
decision  making,  such  as  being  directed  to  cover  a  neighboring  lane  during  a  defensive 
counter  air  (DCA)  lane  defense  mission,  finding  out  that  a  member  of  your  4-ship  has  no 
radar  capabilities,  or  losing  Airborne  Warning  and  Control  System  (AWACS)  support. 
What-if  events  will  be  stored  in  a  relational  database  and  will  be  assigned  attributes 
including  approximate  difficulty  level  and  category/ies  of  mission  knowledge 
emphasized.  Additional  attributes  may  be  assigned  if  Phase  II  research  identifies 
additional  elements  of  expertise  that  event  presentations  should  be  structured  to  target. 
The  list  of  what-if  events  presented  to  the  user  will  be  randomly  selected  without 
replacement  from  a  subset  of  what-if  events  that  are  consistent  with  the  approximate 
level  of  expertise  suggested  by  his  user  profile  and  progress  through  the  DMT  syllabus. 
Because  this  particular  user  is  relatively  inexperienced  and  is  preparing  for  his  first  DMT 
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mission,  the  list  of  events  he  is  shown  will  be  relatively  easy  events  that  emphasize  the 
use  of  more  basic  types  of  mission  knowledge,  such  as  knowledge  required  to  establish 
decision  criteria  and  knowledge  required  to  establish  timing  and  flow. 

Selecting  what-if  challenges.  The  user  may  develop  a  plan  for  a  given  what-if 
event  and  then  return  to  the  list  of  events  to  choose  another.  Alternatively,  the  user 
may  choose  to  be  given  one  or  more  what-if  challenges  after  choosing  a  what-if  event. 
What-if  challenges  are  situations  that  might  occur  within  the  context  of  the  what-if  event 
as  it  unfolds.  As  an  example,  a  user  who  selected  the  what-if  event  ‘3-Group  Wall’ 
might  be  given  what-if  challenges  such  as  those  in  the  following  list: 

?  Southern  bandit  drags  south 

?  Southern  bandit  turns  back  in  but  Viper  03  previously  broke  lock 
?  Viper  03  detects  Southern  bandit  hot 

?  Viper  03  can’t  get  clear  channel  to  communicate  change  in  bandit’s  beari  ng  and 

aspect 

?  Viper  03  decides  to  engage  without  communicating  decision 
?  Viper  03  exits  merge  and  wants  to  find  Viper  04 

?  Viper  03  trying  to  find  04,  flies  too  close  to  engagement  and  gets  a  buddy  spike 
?  Viper  04  targeting  northern  group  instead  of  middle 
?  Viper  02  isn’t  making  targeting  call  when  you  expect  to  hear  it 
?  Vipers  exit  merge 

?  Viper  02  flies  in  too  far  and  gets  mud  spike 

?  Cold  Ops:  There  are  four  remaining  bandits  and  Viper  02  and  04  are  low  on  fuel 
?  Cold  Ops:  Jamming  interferes  with  AWACS  communications 
?  Cold  Ops:  Viper  02  reports  bingo 

What-if  challenges  will  be  stored  in  a  database  and  will  be  assigned  attributes 
including  what-if  events  for  which  they  are  appropriate,  event  phase  with  which  they  are 
associated,  approximate  level  of  difficulty,  and  category/ies  of  mission  knowledge 
emphasized.  A  new  what-if  challenge  will  be  presented  to  the  user  each  time  he  clicks 
on  the  ‘New  What-lf  Challenge’  button.  Challenges  tend  to  be  associated  with  event 
phases  and  so  will  be  presented  in  a  serial  order  that  is  consistent  with  progression 
through  those  phases.  Each  challenge  presented  will  have  been  randomly  selected 
without  replacement  from  a  subset  of  what-if  challenges  that  are  matched  to  the  current 
what-if  event,  event  phase,  approximate  level  of  user  expertise,  and  progress  through 
the  DMT  syllabus. 

The  what-if  planning  environment.  Once  the  user  has  selected  a  what-if 
event,  he  enters  the  what-if  planning  environment.  A  depiction  of  the  proposed 
conceptual  design  of  this  environment  is  shown  in  Figure  2.  The  environment  includes 
a  central  snapshot  panel,  a  supporting  information  panel  to  the  right  of  the  snapshot 
panel,  a  knowledge  categories  panel  on  the  left  side  of  the  snapshot  panel,  and  a 
toolbar  across  the  bottom  of  the  screen.  When  the  user  enters  this  environment,  the 
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snapshot  panel  will  contain  symbols  (e.g.,  arrows)  showing  the  headings  and  positions 
of  hostile,  bogey,  and  friendly  aircraft  associated  with  the  scenario  and  the  selected 
what-if  event.  A  summary  of  the  what-if  event  will  be  presented  as  a  heading  at  the  top 
of  the  screen. 


Figure  2.  Phase  I  design  of  the  C3PI  what-if  planning  environment. 


Using  the  planning  environment.  To  develop  a  plan  in  response  to  a  given 
what-if  event,  the  user  “pens”  to  interact  with  the  electronic  whiteboard.  Specifically,  the 
user  can  draw  and  erase  colored  lines  and  symbols  on  the  snapshot  panel.  He  can 
additionally  write  supporting  details,  such  as  ranges  and  altitudes,  on  the  snapshots. 
Planning  support  and  training  tools  can  be  used  to  supplement  and  guide  plan 
development,  and  by  clicking  on  the  Knowledge-Use  box,  the  user  will  have  a  set  of 
knowledge  categories  from  which  planning  prompts,  or  knowledge-use  prompts,  may  be 
requested. 

Supporting  information  panel.  The  supporting  information  panel  provides 
another  location  for  planning  information  to  be  recorded.  This  panel  is  intended  to  allow 
the  user  to  write  down,  type  in,  or  copy  and  paste  from  a  previous  plan  considerations, 
decisions,  and  guidance  specific  to  a  plan  developed  for  a  given  what-if  event  and 
challenge.  The  supporting  information  contained  in  this  panel  will  be  saved  together 
with  the  associated  snapshot  if  the  user  decides  to  save  his  plan.  We  propose  to 
determine  the  exact  set  of  supporting  information  categories  that  users  may  choose  to 
work  by  conducting  additional  assessments  of  user  requirements  during  Phase  II. 
Examples  of  supporting  information  categories  include  the  following: 
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?  Challenges  and  Risks  -  User  can  list  potential  problems,  challenges,  and  risks 
associated  with  each  what-if  challenge. 

?  Decision/Tactic  and  Criteria  -  User  can  list  decision  points  and  tactics  and  their 
associated  criteria.  Examples  of  decision  points  include  split  points,  intercept 
points,  and  abort  decisions. 

?  Player  and  Comm  -  User  can  list  examples  of  radio  calls  that  should  be  made  by 
the  AWACS,  F-16  pilots,  and  any  other  event  participants,  or  list  comm  discipline 
rules  that  should  be  followed  (e.g.,  who  has  comm  priority  during  the  event). 

?  Keys  to  Success  -  User  can  list  ‘Keys  to  Success’,  a  set  of  reminders  about  the 
key  things  that  need  to  be  accomplished  by  the  team  for  the  event  or  mission  to 
be  successful. 

?  Relevant  Lessons  Learned  -  User  can  review  lessons  learned  identified  by  her 
team  during  previous  missions  by  clicking  on  the  ‘Lessons  Learned’  button.  Of 
these  lessons  learned  (they  can  also  use  other  available  sources),  the  user  might 
want  to  list  any  that  might  be  relevant  to  the  current  event.  This  might  increase 
the  influence  of  lessons  learned  on  planning  and  it  might  be  useful  information  to 
include  in  the  mission  brief. 

Knowledge  categories  and  knowledge-use  prompts.  The  user  will  be 
encouraged  to  use  the  knowledge  categories  panel  to  support  the  what-if  planning 
process  (the  encouragement  mechanism  will  be  determined  by  working  with  users 
during  Phase  II).  These  knowledge  categories  are  heavily  influenced  by  types  of 
knowledge  members  of  naval  air  wing  tactical  teams  were  found  to  use  to  achieve  team 
coordination  in  a  CTA  of  naval  strike  team  coordination  (Neville  et  al.,  2002).  The  CTA 
additionally  revealed  that  certain  knowledge  categories  were  used  infrequently  by  less 
experienced  team  members.  For  example,  knowledge  about  other  team  member’s 
procedures  and  capabilities  and  about  one’s  own  limitations  and  constraints  tended  to 
not  be  used  by  less  experienced  team  members.  In  addition,  although  experienced  and 
less  experienced  team  members  appeared  to  be  equally  knowledgeable  about  team 
member  support  requirements,  the  less  experienced  team  members  were  less  likely  to 
describe  using  that  knowledge  to  anticipate  and  proactively  meet  support  requirements. 

Accordingly,  C3PI  knowledge  categories  labeled  ‘limitations’  and  ‘mutual  support’ 
are  considered  more  advanced  than  those  for  which  experience-related  differences 
were  not  observed,  such  as  ‘decision  points  and  criteria’  and  ‘timing  and  flow’.  The  C3PI 
knowledge  category  ‘Initial  SA’  was  added  based  on  Phase  I  interviews  indicating  that 
an  important  part  of  planning  for  what-if  situations  is  determining  how  the  situation  will 
be  detected  and  communicated  to  the  team.  Another  knowledge  category,  ‘enemy 
tactics’  was  added,  based  on  data  collected  during  both  the  CTA  of  naval  strike  team 
coordination  (collected  but  not  part  of  the  analysis)  and  C3PI  Phase  I  interviews.  These 
data  and  interviews  suggested  that  knowledge  about  enemy  tactics  is  associated  with 
expertise,  and  consequently  ‘enemy  tactics’  was  added  to  the  C3PI  knowledge 
categories  and  is  considered  a  more  advanced  category.  The  proposed  Phase  II 
research  will  examine  whether  knowledge  about  building  initial  SA  and  enemy  tactics  is 
brought  to  bear  more  often  or  differently  by  more  versus  less  experienced  air  combat 
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team  planners.  In  addition,  it  will  reexamine  the  other  knowledge  categories  with 
respect  to  experience-related  differences. 

To  use  these  knowledge  categories,  the  user  clicks  on  them,  one  at  a  time,  to 
access  a  list  of  knowledge-use  prompts.  The  user  then  selects  a  knowledge-use 
prompt  for  a  given  category  and  that  prompt  appears  in  a  text  box  inside  the  knowledge 
category  panel.  The  prompts,  which  will  be  stored  in  a  database,  are  designed  to 
prompt  the  user  to  consider  the  types  of  information  and  use  the  types  of  critical  thinking 
strategies  an  expert  what-if  planner  might  use.  They  represent  a  form  of  attentional 
advice,  a  training  strategy  whereby  trainees  are  given  guidance  about  how  to  focus  their 
attention  during  a  training  session  (e.g.,  Cannon-Bowers  et  al.,  1 998). 

Because  evidence  suggests  that  the  knowledge  categories  may  differ  with 
respect  to  how  fluently  or  adeptly  they  can  be  used  by  less  experienced  members  of 
tactical  teams,  the  knowledge  category  panel  will  be  designed  to  guide  trainees’  use  of 
these  categories.  Further,  in  case  a  user  lacks  the  knowledge  to  respond  effectively  to 
a  given  knowledge-use  prompt,  each  prompt  will  be  associated  with  expert  advice  video 
clips,  portions  of  official  USAF  planning  doctrine,  and  any  relevant  enemy  or  friendly 
platform  capability  specifications.  In  other  words,  each  prompt  will  have  a  direct  link  to 
contents  of  planning  and  training  tools  described  below. 

Each  knowledge  category  will  be  associated  with  multiple  knowledge-use 
prompts  identified  during  the  Phase  I  and  Phase  II  research  efforts.  Example  prompts 
for  each  category  are  as  follows: 

?  Initial  SA 

-  How  will  team  build  SA  about  this  situation? 

-  What  needs  to  be  done  and  said  and  by  whom? 

-  How  will  AWACS  describe  this  configuration  over  the  radio? 

?  Decision  Points  and  Criteria 

-  What  are  the  decisions? 

-  Who  is  the  decision  maker  for  a  given  decision  (e.g.,  AWACS  or  Flight 
Lead)?  Will  decision  maker  need  help  recognizing  that  criteria  were  met? 

-  Are  the  decision  criteria  consistent  with  the  risk  acceptance  level? 

-  What  could  team  members  do  or  say  to  prevent  the  what-if  situation  from 
reaching  abort  (or  other  negative)  criteria? 

-  Are  there  steps  you  can  take  to  build  an  adaptive  mindset  to  facilitate  the 
required  shift  in  the  plan  when  criteria  are  met? 

?  Timing  and  Flow 

-  Who  should  be  where?  How  should  they  be  flowing? 

?  Mutual  Support 

-  Who  needs  support  during  this  situation? 

-  How  will  the  support  need  be  recognized? 

-  Who  will  provide  the  needed  support  and  how? 

?  Enemy  Tactics 

-  What  might  the  enemy  do  next  in  this  situation? 
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-  How  can  you  be  best  positioned  to  respond? 

?  Limitations 

-  What  factors  might  limit  your  ability  to  detect  or  respond  to  this  situation? 

-  What  can  the  team  do  to  counter  identified  limiting  factors? 

The  planning  support  and  training  tools.  In  case  the  user  finds  that  he  lacks 
the  knowledge  to  answer  a  given  knowledge  use  prompt,  each  prompt  will  be  directly 
linked  with  relevant  C3PI  planning  and  training  tools.  Thus,  for  example,  if  more 
information  about  enemy  or  team  member  platform  (e.g.,  AWACS)  capabilities  is 
needed  to  respond  to  the  knowledge-use  prompt,  the  user  will  be  able  to  jump  directly 
from  the  prompt  to  the  relevant  information  contained  in  the  Platform  Specs  tool. 
Alternatively,  if  official  USAF  planning  guidance  is  relevant,  a  direct  link  to  the  Official 
Guidance  tool  will  be  available.  Similarly,  direct  links  will  be  available  to  take  the  user  to 
Expert  Advice  video  clips  of  other  pilots  describing  a  planning  strategy  they  use  or  a 
mission  consequence  they  experienced  that  is  relevant  to  the  knowledge-use  prompt. 

Planning  support  and  training  tools  can  be  accessed  at  any  time  during  a 
planning  session.  In  addition  to  helping  users  build  knowledge  in  the  C3PI  knowledge 
categories,  they  can  be  used  as  general  sources  of  case-based  training  (e.g.,  through 
expert  advice  video  clips)  and  cross-training  (e.g.,  through  access  to  platform  specs) 
that  may  contribute  to  the  acquisition  of  what-if  mission  planning  expertise.  As  noted 
previously,  planning  support  and  training  tools  may  be  added  during  Phase  II  to  target 
additional  elements  of  what-if  and  contingency  planning  expertise  identified  during  the 
remainder  of  the  Phase  I  research  or  during  the  Phase  II  research.  Currently,  tools 
proposed  for  inclusion  in  the  C3PI  planning  environment  include: 

?  Official  Guidance 

-  Official  planning  guidance,  e.g.,  squadron  planning  guides  and  Air  Force 
Tactics,  Techniques,  and  Procedures  (AFTTP)  3-1 

?  Expert  Advice 

-  Video  clips  in  which  instructor-level  pilots  describe  planning  strategies  that 
have  worked  for  them,  factors  they  have  learned  to  consider,  etc.  (or  role- 
players  describe  the  pilots’  advice) 

-  Advice  taken  from  CTA  interviews  and  other  interview  opportunities 
?  Platform  Specs 

-  Capabilities,  limitations,  and  characteristics  of  friendly  and  enemy  aircraft 
?  Keys  to  Success 

-  Compilation  of  keys  to  success  accumulated  by  a  given  DMT  team  during 
their  week  of  exercises,  organized  by  scenario 

-  Might  include  access  to  keys  to  success  accumulated  by  other  teams 
?  Lessons  Learned 

-  Compilation  of  lessons  learned  identified  during  AAR  by  a  given  DMT 
team  throughout  their  week  of  exercises,  organized  by  DMT  scenario 

-  Might  include  access  to  lessons  learned  of  other  teams 
?  Review  Competencies 
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-  Allows  user  to  review  elements  of  planning  expertise  and  knowledge 
categories  targeted  by  a  given  what-if  event  or  what-if  challenge 

Using  the  what-if  plans  during  briefing  and  AAR.  In  addition  to  supporting  the 
mission  planning  phase,  C3PI  is  designed  to  facilitate  mission  briefing  and  AAR,  as 
shown  in  Figure  3.  For  example,  after  the  user  has  completed  a  plan  for  a  given  what-if 
event  or  what-if  challenge,  she  can  save  it  for  later  use.  During  the  mission  brief,  these 
saved  plans  -  each  consisting  of  a  snapshot  and  the  associated  supporting  information 
(from  the  supporting  information  panel)  -  can  be  printed  and  distributed  to  team 
members  and/or  displayed  using  the  DMT  Collaborative  Planning,  Briefing,  and 
Debriefing  System  (e.g.,  Sidor,  2002),  which  also  utilizes  the  Smart  Technologies 
interactive  whiteboard  system.  If  they  are  displayed  using  the  DMT  Collaborative 
Planning,  Briefing,  and  Debriefing  System,  the  user  will  be  able  to  continue  to  draw  on 
the  plans  during  the  briefing  sessions  and  will  be  able  to  utilize  C3PI  planning  and 
training  tools.  To  facilitate  tool  use  in  this  and  the  AAR  setting,  the  C3PI  will  feature  a 
keyword-based  tool  search  option  so  that  the  user  may  quickly  identify  relevant  expert 
advice,  lessons  learned,  official  guidance,  etc.  to  answer  plan-related  questions  that 
may  arise. 

C3PI  also  allows  snapshots  and  supporting  information  to  be  printed  and 
distributed  to  team  members  during  the  mission  brief.  Mission  plan  snapshots  tend  to 
be  easy  to  comprehend  and  follow,  they  can  help  a  team  develop  a  shared  mental 
model  during  mission  execution,  and  they  may  help  team  members  follow  the  briefed 
plan.  For  these  reasons,  team  members  may  want  to  bring  the  snapshots  into  their 
aircraft  with  them. 


Briefing  charts  supplied 
by  C3PI 


Mission  Briefed 


Events  that  require 
dynamic,  on-the-fly 
adaptation  and  decision 
making  flagged  during 
execution 


Mission  Flown 


C3PI  CTA  contributes 
performance  assessment 
items 


Access  to  C3PI 
whiteboard, 
planning  tools,  snapshots, 
nd  supporting  information. 


After  Action  Review/ 
Debrief 

Debrief  Agent  prompts 
team  to  assess  plan  with 
respect  to  execution  of 
flagged  events;  C3PI 
knowledge  categories  will 
influence  prompts 


Figure  3.  Description  of  ways  C3PI  will  contribute  to  mission  briefing  and  AAR. 
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The  C3PI  what-if  plans  may  be  displayed  using  the  same  collaborative  system 
during  debriefing,  or  AAR.  This  will  allow  a  team  to  view  both  their  DMT  mission 
performance  playback  and  their  relevant  what-if  plans  simultaneously.  Dynamic 
replanning  and  decision  making  events  that  occur  during  mission  execution  -  including 
both  events  that  map  onto  what-iffed  events  and  events  that  were  not  specifically 
addressed  during  what-if  planning  -  could  be  flagged  (e.g.,  using  the  AFRL 
performance  evaluation  and  tracking  system  [PETS])  during  mission  execution  for 
subsequent  assessment  using  the  C3PI  plans  and  tools).  In  particular,  the  C3PI  design 
features  a  team  self-correction  training  tool  -  the  debrief  agent  -  that  prompts  the  team 
to  evaluate  their  preparation  for  and  performance  during  dynamic  replanning  and 
decision  making  events  in  a  constructive  and  process-oriented  way  (e.g.,  Blickensderfer 
et  al.,  1997).  Teams  will  additionally  be  prompted  to  add  any  lessons  learned  to  C3PI 
for  use  in  their  future  C3PI  planning  sessions  (access  to  team  plans,  data,  and  inputs 
will  be  limited  to  team  members  and  DMT  instructors). 

C3PI  System  Specifications 

This  section  describes  the  C3PI  system  and  application  requirements.  The  C3PI 
Phase  II  plan  proposes  incremental  development  of  the  system  and  application  based 
or  iterative  reviews  and  refinements  with  the  user  community.  The  system  is  described 
in  general  terms  because  it  is  expected  to  evolve  as  Phase  II  progresses. 

Hardware  configuration.  The  C3PI  system  will  be  implemented  with  a  custom 
software  application  executing  on  a  standard  Intel-based  Personal  Computer.  The  C3PI 
system  will  incorporate  a  SMART  Board™  unit  developed  by  SMART  Technologies,  Inc. 
This  device  emulates  a  writing  surface  similar  to  that  of  a  dry-erase  whiteboard  by  using 
a  coordinate  sensing  display  overlay  and  a  special  stylus.  An  application  screen  is 
projected  on  the  SMART  Board.  Images  are  drawn  with  a  special  ‘color’  stylus  retrieved 
from  several  stored  in  the  SMART  Pen  Tray™.  The  SMART  Pen  Tray  communicates 
with  the  whiteboard  emulation  application  so  that  the  application  can  duplicate  the 
freehand  drawing  in  the  color  selected  by  the  user.  The  integrated  stylus  may  also 
emulate  standard  mouse  input  for  the  application  that  is  driving  the  projected  display. 
This  allows  the  user  to  interact  with  the  application  as  one  would  with  a  desktop 
application. 

The  C3PI  PC  will  use  a  10/100  Base-T  Ethernet  Network  Interface  Card  (NIC)  as 
an  interface  to  a  LAN  or  WAN.  This  hardware  will  support  the  Microsoft  NetMeeting™ 
application  in  providing  distributed  training  capability  for  C3PI. 

Network  interoperability.  The  C3PI  system  will  be  capable  of  remote 
interoperability  using  a  LAN  or  WAN  connection.  The  Microsoft  NetMeeting™ 
application  will  be  used  to  transparently  provide  this  interoperability.  Microsoft 
NetMeeting™  is  a  powerful  teleconferencing  and  collaboration  network  application. 
NetMeeting  supports  the  traditional  teleconferencing  capabilities  such  as  bi-direction 
audio  and  video  feeds,  chat  and  whiteboard  interactions  and  file  transfers.  C3PI  benefits 
from  NetMeeting’s  ability  to  reproduce  a  PC  display  at  a  remote  site  and  receive 
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application  interactions  from  users  at  those  remote  sites.  NetMeeting  communicates 
using  TCP/IP  so  integration  with  a  secure  network  is  trivial  since  TCP/IP  may  be 
wrapped  in  a  secure  transport- layer  envelope. 

SMART  Board  whiteboard  emulation.  The  SMART  Board™  includes  utility 
software  to  allow  capture  of  the  ‘whiteboard’  drawn  images.  This  utility  will  be  used  to 
capture  scenario  drawings,  save  them  with  the  scenario,  and  later  restore  the  images  so 
that  they  may  overlay  the  application-generated  graphics. 

Application  context:  The  data  model.  The  C3PI  data  model  represents  a 
session  that  is  saved  as  a  standard  operating  -system  file.  The  data-model  contains  all 
context  necessary  to  preserve  (save)  and  reconstruct  (load)  the  session.  The  data 
model  is  hierarchical  where  the  topmost  ‘root’  element  is  the  team.  A  C3PI  session  is 
organized  with  respect  to  the  team  members  -  users.  For  a  particular  user,  the  data 
model  contains  a  list  of  sessions  for  which  the  user  has  participated.  Each  user  is  also 
associated  with  training  statistics  and  history  describing  overall  C3PI  participation.  The 
data  model  will  evolve  as  the  C3PI  application  context  progresses  through  its  iterative 
development  cycle.  The  C3PI  data  model  specifies  team  profiles,  sessions,  scenarios, 
what-if  events,  what-if  challenges  and  is  abstracted  as  follows: 

Team  profile.  The  team  profile  corresponds  to  a  unique  file  name.  When  a  C3PI 
session  is  initiated,  the  user  will  create  a  new  team  profile  file,  or  open  an 
existing  predefined  team  profile  file.  The  team  profile  contains  a  C3PI  historical 
record  for  the  team  and  a  list  of  User  Profile  Identifiers.  Each  user  profile  will 
contain  the  team  member’s  unique  C3PI  history,  various  statistics,  and  a  list  of 
the  performed  sessions  and  their  respective  context: 


Figure  4.  Team  Profile  Data  Model 
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Session.  Each  session  for  a  given  team  member  will  contain  a  particular 
syllabus.  The  syllabus  will  be  comprised  of  multiple  DMT  scenarios: 


Figure  5.  Session  Data  Model 


Scenario.  Each  DMT  scenario  will  contain  a  link  to  the  team’s  DMT  history  and  a 
link  to  the  mission  synopsis.  These  items  are  represented  as  links  as  their 
context  does  not  vary  for  each  scenario.  A  list  of  what-if  events  and  their 
respective  context  will  be  maintained  in  the  scenario  unique  to  the  particular  user 
and  session. 


Figure  6.  Scenario  Data  Model 

What-if  Event.  Each  what-if  event  will  contain  a  link  to  its  associated  whiteboard 
overlay.  This  overlay  will  be  created  and  revised  by  the  user  and  merged  with 
static  or  interactive  background  graphics  by  the  C3PI  application  and  COTS 
software  provided  with  the  SMART  Board.  Each  event  will  define  unique  text  that 
may  be  modified  by  the  user  in  supporting  information  panels.  Each  event  will 
also  contain  links  to  what-if  challenges  that  may  be  presented  during  the  event. 
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Figure  7.  What-if  Event  Data  Model 


What-if  Challenge.  Each  what-if  challenge  will  define  a  particular  difficulty  level, 
mission  phase  and  the  emphasized  knowledge  category  for  the  corresponding 
event. 


Figure  8.  What-if  Challenge  Data  Model 


Application  Navigation:  the  Behavior-Model.  The  C3PI  application  context  is 
defined  in  the  time  domain.  As  the  training  session  proceeds,  the  application  will 
progress  from  the  ‘current  state’  to  a  logically  successive  ‘next  state’.  The  user  may 
intervene  and  interact  with  the  application  independent  of  the  state  sequence.  The  user 
may,  at  appropriate  stages  in  the  application,  invoke  specific  functions  such  as 
displaying  expert  advice  video  clips. 

These  application  flows  will  be  captured  and  represented  as  a  behavior-model. 
The  transitions  through  the  application  states  will  be  invoked  by  the  user.  This  does  not 
preclude  the  ability  to  support  autonomous  transitions,  such  as  timed  exercises  or  slide- 
show  presentations.  These  transitions  may  not  necessarily  compose  a  single-thread 
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sequential  sequence,  but  may  require  that  the  user  branch  or  navigate  back-and-forth 
through  the  session. 

The  user  may  choose  to  deviate  from  the  ‘nominal’  session  sequence  by 
interacting  explicitly  with  the  application.  The  nominal  sequence  will  be  internally 
represented  in  the  application.  This  facilitates  ease  of  use  in  that  the  user  does  not  need 
to  progress  through  the  sequences  by  opening  files  and  pressing  specific  buttons.  The 
nominal  session  sequence  is  illustrated  in  Figure  9. 


(  Initiate  Session  )— ►{  Create  User  Profile  )— ►{Choose  Syllabus } — 

~K 

'C  Select  Team  'Y\ 

^»{  Enter  Flight  Hours'")-^ 

Enter  Sim  Hours 


►{Choose  Scenario-) - ►{  Show  Mission  Synopsis  {r 


Develop  Mission  Plan 

— ►{  Show  What-if  Events  ) — <{  ►{RutT) 

Select  Mission  Plan  3 — ' 


Figure  9.  Nominal  Session  Sequence 


User  Interface.  The  user  interface  design  presents  the  typical  challenges 
incurred  when  presenting  interactive  information  and  control  elements  on  a  graphical 
screen  of  limited  size  and  effective  resolution.  Upon  considering  the  requirements  of 
the  C3PI  system,  the  user  interface  may  be  decomposed  into  four  functional  areas: 

?  Scenario  Pane:  A  large  region  for  displaying  static  and  interactive  graphics  and 
whiteboard  emulation. 

?  Video  popup  Window:  A  temporary  region  for  displaying  video  clips. 

?  Supporting  Information  Pane:  A  region  for  presenting  dynamic  runtime-variant 
information. 

?  Control  Tool  Bar:  A  region  used  to  present  frequently  used  navigation  controls. 

There  exist  opposing  forces  in  arriving  at  an  optimal  user  interface.  For  example, 
it  is  a  foregone  conclusion  that  the  C3PI  application  will  use  the  graphical  interface 
provided  by  the  Microsoft  Windows  desktop  operating  system  due  to  other  design 
drivers.  The  guidelines  for  effective  use  of  this  interface,  provided  to  Microsoft 
application  developers,  prescribes  consistent  use  of  the  various  graphical  etements 
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(buttons,  edit-boxes,  lists,  etc.)  whenever  possible.  Applications  designed  in 
compliance  with  these  guidelines  provide  an  intuitive  look-and-feel  and,  therefore, 
require  less  training  as  the  user  community  matures  on  the  Microsoft  Windows  platform. 
The  opposing  force  is  due  to  considerations  when  contemplating  the  usability  of  C3PI. 
Those  recommendations  prescribed  by  Microsoft  may  be  deprecated  when  considering 
particular  use  cases  of  the  C3PI  application,  leading  to  evaluation  of  alternate 
approaches.  The  initial  C3PI  application  screen  concept  is  illustrated  Figure  10.  This 
concept  represents  the  straw-man  approach  that  will  be  used  to  iteratively  design  the 
optimal  user  interface. 


C3PI  -  <session  name>  -  <sequence  name> 


File  Xools  Help 


Knowledge-Use  Sfl 

How  will  you  gain  SA  in  this 
situation  ? 


Challenges  and  Risks 

Descision/Tactic  Criteria 

, 

Official 

Guidance 

Expert 

Advice 

Platform 

Specs 

Keys  to 
Success 

Review 

Competencies 

Review 

Snapshots 

New  What- If 
Challenge 

New  What-lf 
Event 

Event:  3-Group  Wall 

Challenge:  Enemy  begins  maneuvering,  shots  trashed. 

Figure  10.  Evolved  Application  Screen  Concept 


RESEARCH  CONTRIBUTIONS 


This  section  of  the  report  describes  domain  analysis  research  conducted  using 
CTA  methods  to  guide  the  C3PI  design  and  research  issues  and  plans  that  will  be 
pursued  in  Phase  II  of  this  research  and  development  effort. 


Phase  I  CTA  Research 


To  support  the  development  of  the  C3PI  conceptual  design,  CTA-based  research 
was  conducted  during  Phase  I.  This  research  had  the  goals  of  exploring  the  links 
between  planning  and  mission  execution  and  contributing  to  the  development  of  training 
content  to  be  used  in  C3PI.  For  this  research,  an  interview  protocol  was  developed  to 
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support  an  event-based  CTA  approach  (e.g.,  Fowlkes,  Salas,  Baker,  Cannon-Bowers,  & 
Stout,  2000).  We  used  a  CTA  approach  for  three  main  reasons.  First,  this  approach 
facilitates  the  identification  of  rich  task  performance  details  and  provides  a  means  for 
representing  those  details  in  ways  that  reveal  important  relationships  and  patterns  in 
skill  and  knowledge  usage.  Second,  the  rich  task  performance  details  and  knowledge 
representations  obtained  using  this  type  approach  can  support  the  development  of  a 
number  of  training-related  products  including  performance  measures,  training 
scenarios,  performance  critiquing  tools,  training  and  performance  aids,  and  feedback 
content.  Third,  the  C3PI  project  team  has  significant  experience  conducting  CTAs  of 
this  type.  The  CTA  included  a  front-end  domain  analysis  and  subsequent  development 
and  implementation  of  the  event-based  interview  protocol.  These  are  described  below. 

CTA  techniques  can  yield  a  large  amount  of  very  detailed  information.  So  that 
the  analyst  does  not  end  up  transcribing,  representing,  assessing,  and  ultimately 
filtering  out  large  amounts  of  interesting  but  irrelevant  data,  it  is  essential  thatthe  CTA 
technique(s)  target  the  correct  information.  Therefore,  we  initially  performed  a  front-end 
domain  analysis.  This  analysis  included  documentation  review  and  unstructured 
interviews.  For  the  documentation  review,  we  drew  upon  the  mission  planning  and 
aircrew  training  literature  (especially  reviews,  e.g.,  Blickensderfer  et  al.,  2000;  Cannon- 
Bowers,  Tannenbaum,  Salas,  &  Volpe,  1995;  Nullmeyer  et  al.,  2000),  including  naval  air 
wing  team  coordination  research  we  currently  are  conducting,  to  identify  knowledge  and 
skill  categories  that  have  been  proposed  and/or  evaluated  in  past  research  efforts.  We 
also  made  use  of  reports  and  data  generated  by  the  operational  community  and  DMT 
research  to  obtain  insight  into  aspects  of  mission  performance  that  are  affected  by 
planning  and  to  further  assess  the  skills  and  knowledge  used  to  effectively  plan  during 
mission  execution. 

Second,  the  front-end  analysis  was  supported  by  high-level  interviews  with 
aircrew  members.  Specifically,  two  background  interviews  were  conducted  (with  one 
SME)  to  provide  a  better  understanding  of  DCA  missions,  including  the  planning  and 
briefing  processes  associated  with  them.  These  interviews,  as  well  as  the 
documentation  review,  were  utilized  to  develop  the  event-based  interview  protocol  that 
was  subsequently  implemented  with  two  SMEs  (the  SME  used  in  the  initial  interviews 
and  an  additional  SME). 

The  intent  of  the  event-based  interview  was  to  provide  experts  with  deliberate 
and  controlled  job  situations,  allowing  investigation  of  specific  task  aspects  and  the 
comparison  of  expert  responses.  The  event-based  interview  protocol  involved 
presenting  SMEs  with  tactical  situations  that  would  provide  rich  contexts  within  which 
they  could  describe  links  between  planning  and  execution.  This  protocol  allowed  us  the 
opportunity  to  identify  important  cues,  knowledge,  and,  importantly,  links  between 
execution  and  the  planning  process.  Further,  because  different  SMEs  will  each  have 
experienced  somewhat  different  situations,  tried  different  strategies,  and  used  different 
cues  and  data,  and  because  different  SMEs  will  recall  and  verbalize  information 
differently,  presenting  multiple  SMEs  with  the  same  event  and  amassing  data  from  all  of 
them  that  is  specific  to  that  event  is  critical  to  building  a  complete  and  robust  picture  of 
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task  performance  and  its  cognitive  elements.  Table  1  provides  examples  of  the  event 
prompts  that  were  used  in  this  interview  protocol.  The  complete  interview  protocol  is 
provided  in  Appendix  A. 


Table  1 .  Example  E vent-Based  Interview  Prompts 

•  During  the  CAP  (combat  air  patrol)  phase,  you  find  out  that  #2  and  #3  have 
an  alibi  -  neither  has  working  radar. 

•  During  the  merge  preparation  phase,  #3  makes  confusing  sorting  comms  as 
you  sort  against  4  hostile  aircraft  and  you  are  not  sure  if  he  is  targeting  the 
one  you  wanted  him  to  target. 

•  Hostile  aircraft  have  been  trying  to  engage  you  in  cat  and  mouse  game. 
Fighters  in  the  “next  lane"  have  fallen  out  so  now  you  have  to  cover  their  lane. 


Two  SMEs  participated  in  the  event-based  interviews.  For  each  situation 
presented,  each  SME  was  prompted  to  recall  specific  situations,  similar  to  the  events 
presented,  with  which  they  had  personal  experience.  The  SMEs  were  also  prompted  to 
link  the  execution  experiences  recounted  to  the  planning  and  briefing  process.  This 
interview  technique  is  a  variation  of  the  Critical  Decision  Method  (CDM;  e.g.,  Klein, 
Calderwood,  &  MacGregor,  1989)  that  facilitates  the  comparison  of  SME  responses  and 
the  accumulation  of  SME  responses  for  a  given  set  of  prompts.  CDM  is  designed 
specifically  to  access  knowledge  that  accompanies  expertise  and,  as  a  case-based 
technique,  takes  advantage  of  human  tendencies  to  use  storytelling  as  a  means  of 
conversing  and  communicating  and  to  use  stories  as  a  means  for  storing  and  describing 
memories  (e.g.,  Cohen,  Freeman,  &  Thompson,  1998;  Jonassen,  Tessmer,  &  Hannum, 
1999). 


Each  of  the  interviews  was  synthesized  to  extract  key  processes  in  planning  and 
execution.  These  key  processes  were  used  as  categories  for  assessing  the  data  and 
are  summarized  in  Table  2.  Specifically,  interview  data  were  parsed  into  these  key 
planning  and  execution  process  categories  and  then  coded  with  respect  to  the 
knowledge  categories  listed  in  Table  3.  These  knowledge  categories  were  derived 
based  on  iterative  assessment  of  the  data  collected  and  CTA  research  conducted  with 
naval  air  wing  aviators  (Neville  et  al.,  2002),  and  they  closely  correspond  to  C3PI 
knowledge  categories  and  knowledge-use  prompts.  Thus,  knowledge  verbalized  during 
these  interviews  can  be  used  to  support  the  development  of  these  categories  and  the 
associated  training. 

The  interview  analysis  syntheses  are  included  as  Appendix  B.  In  summary,  the 
event-based  interviews  conducted  in  Phase  I  served  to  increase  our  domain  knowledge 
and  help  us  to  identify  the  links  between  planning  and  execution.  Further,  we  anticipate 
that  they  will  be  used  to  develop  specific  C3PI  training  content  during  Phase  II.  In 
addition,  this  effort  resulted  in  an  interview  protocol  that  may  be  utilized  in  the  Phase  II 
effort.  One  of  the  two  background  interviews  conducted  as  part  of  the  front-end 
analysis  was  recorded  and  transcribed  and  was  also  assessed  using  the  categories  and 
codes  listed  in  Tables  2  and  3.  These  interview  data  are  included  as  Appendix  C. 
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Table  2.  Categories  of  Information  Extracted  from  Interviews 

Category  of  Information 

Category  Description 

Planning  elements  and 
organization 

Interviews  were  analyzed  to  identify  key 
elements  of  the  planning  process  and  identify 
how  the  planning  and  briefing  processes  are 
organized. 

Planning  strategies  that 
support  execution 

The  interviews  were  synthesized  to  reveal 
planning  strategies  that  support  execution. 
Examples  of  strategies  identified  included  use 
of  simulation/role  playing  to  ensure  team 
members  understand  communication,  targeting 
and  sorting;  use  of  visualization  to  build  a 
similar  mental  picture  among  team  members; 
and  use  of  standardization  and  contracts. 

Variations  in  planning  and 
briefing 

The  interviews  were  synthesized  to  identify 
ways  in  which  teams  may  differ.  For  example, 
one  SME  identified  “what-iffing”  as  a  strategy 
used  by  good  teams.  Another  example 
concerned  differences  in  the  briefing  style  of  the 
flight  lead. 

Execution  challenges 

Execution  challenges  were  identified  to  better 
understand  how  planning  may  help  to  overcome 
them.  Examples  of  execution  challenges 
included  communication,  maintaining  situation 
awareness  going  into  the  merge,  and  radar 
mechanics. 

Execution  strategies 

Execution  strategies  were  identified  to  better 
understand  how  planning  may  help  to  overcome 
them.  Examples  of  execution  strategies 
included  methods  for  conducting  “cold  ops,”  and 
ways  to  task  wingmen. 

Knowledge  that  supports 
execution  and  planning 

Knowledge  that  supports  execution  was 
identified.  Knowledge  identified  pertained  to 
knowledge  of  other  elements,  enemy  threat 
capabilities,  and  examples  of  execution 
templates,  for  example.  This  information  will 
help  the  team  to  further  understand  the 
knowledge  that  supports  execution  and  that 
links  planning  to  execution. 

What  if  examples 

Additional  what-if  examples  were  identified  that 
might  form  the  basis  for  “what  if”  events  and 
challenges  included  in  C3PI. 
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Table  3.  Categories  of  Knowledge  Used  to  Code 
Information  Extracted  from  Interviews 


Initial  SA  (SA) 

-  How  will  team  build  SA  about  this  situation? 

-  What  needs  to  be  done  and  said  and  by  whom? 

-  How  will  AWACS  describe  this  configuration  over  the  radio? 

Decision  Points  and  Criteria  (DPC) 

-  What  are  the  decisions? 

-  Who  is  the  decision  maker  for  a  given  decision  (e.g.,  AWACS  or  Flight 
Lead)?  Will  decision  maker  need  help  recognizing  that  criteria  were  met? 

-  Are  the  decision  criteria  consistent  with  the  risk  acceptance  level? 

-  What  could  team  members  do  or  say  to  prevent  the  what-if  situation  from 
reaching  abort  (or  other  negative)  criteria? 

-  Are  there  steps  you  can  take  to  build  an  adaptive  mindset  to  facilitate  the 
required  shift  in  the  plan  when  criteria  are  met? 

Timing  and  Flow  (TF) 

-  Who  should  be  where?  How  should  they  be  flowing? 

Mutual  Support  (MS) 

-  Who  needs  support  during  this  situation? 

-  How  will  the  support  need  be  recognized? 

-  Who  will  provide  the  needed  support  and  how? 

Enemy  Tactics  (ET) 

-  What  might  the  enemy  do  next  in  this  situation? 

-  How  can  you  be  best  positioned  to  respond? 

Limitations  (LM) 

-  What  factors  might  limit  your  ability  to  detect  or  respond  to  this  situation? 

-  What  can  the  team  do  to  counter  identified  limiting  factors? 

Knowledge  of  Distributed  Teams  Capabilities  (DTC) 

(GCI,  F-18s,  F-5s...) 

Knowledge  of  4-Ship  (K4) 

(Load-Out,  Gas,  Capabilities,  Limitations,  Roles,  Responsibilities...) 

Past  Experience  (PE) 

Mission  Execution  Elements  (MEE) 

(Bounding  Range,  Package  Criteria...) 


Hypotheses  about  the  Cognitive  Elements  of  Planning  Expertise 


During  Phase  II  of  this  effort,  we  propose  to  conduct  additional  research  to 
explore  the  planning  strategies  and  skills  associated  with  effective  team  adaptability  in 
dynamic  environments.  More  specifically,  we  propose  to  conduct  CTA-based  research 
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to  examine  the  processes  by  which  flight  leads  with  different  levels  of  experience  and 
their  teams  conduct  the  what-iffing  and  contingency  planning  phase  of  plan 
development.  The  research  will  be  used  to  examine  planning  process  data  and  look  for 
differences  associated  with  experience  level  in  cognitive  processing  activities  (e.g., 
situation  assessment,  analogical  reasoning,  analytical  decision  making,  option 
generation,  etc.),  in  the  use  of  cognitive  strategies  (e.g.,  prioritization,  memory 
enhancement,  and  decision  making  strategies),  and  in  the  use  of  knowledge  throughout 
the  what-iffing  and  contingency  planning  phase  of  planning. 

This  research  is  designed  to  answer  questions  such  as:  What  strategies  and 
types  of  knowledge  do  experts  use  to  question  assumptions?  to  build  shared  mental 
models?  to  identify  problem  areas?  In  addition,  this  research  will  address  planning 
process  hypotheses  suggested  by  the  cognitive  research  literature.  A  sample  of  these 
hypotheses  includes  the  following: 

?  If  experts  have  more  and  richer  domain  knowledge  of  certain  types  (e.g.,  Doane, 
Pellegrino,  &  Klatsky,  1 990;  Neville  et  al.,  2002),  when  and  in  what  ways  do  they 
use  those  types  of  knowledge  to  more  effectively  what-if  and  plan  for 
contingencies? 

?  Do  expert  planners  tend  to  engage  in  a  more  thorough  analysis  of  the  what-if 
situation  and  constraints  before  trying  to  develop  solutions,  as  has  been 
demonstrated  in  other  domains  (e.g.,  Chase  &  Simon,  1 973;  Chi,  Feltovich,  & 
Glaser,  1 981 )?  If  so,  what  are  the  characteristics  of  this  preparatory  analysis  and 
what  aspects  of  it  represent  challenges  for  novices? 

?  A  cognitive  model  of  planning  proposed  by  Hayes-Roth  and  Hayes-Roth  (1979) 
suggests  that  planning  is  characterized  by  opportunistic  decision  making  and 
strategic  alternation  among  levels  of  abstraction. 

-  Are  there  expert-novice  differences  in  the  amount  or  quality  of 
opportunistic  decision  making  during  what-if  planning? 

-  Are  there  expert-novice  differences  in  the  use  of  abstraction  levels  during 
the  what-if  and  contingency  planning  process  that  are  associated  with 
mission  performance?  Research  conducted  by  Spiker  and  his  associates 
(e.g.,  Spiker  &  Nulimeyer,  1 995;  Spiker,  Nullmeyer,  &  Tourville,  1 997) 
suggests  such  differences  may  exist.  For  example,  they  found  that 
superior  plans  tend  to  be  planned  ‘from  big  to  small’  and  that  effective 
planners  are  better  able  to  determine  the  appropriate  level  of  detail. 

?  Analogical  reasoning  has  been  shown  to  be  an  effective  and  commonly-used 
form  of  problem  solving  (e.g.,  Chen,  2002;  Gentner,  1 983)  and  it  is  relevant  to 
the  planning  domain.  For  example,  Spiker  et  al.  (1999)  noted  that  effective 
planners  drew  upon  past  experiences  during  planning.  Flow  and  when  do 
experts  versus  novices  use  analogy  to  guide  planning  during  what-iffing  and 
contingency  planning? 

?  Memory  research  has  shown  that  information  processing  and  presentation 
strategies  affect  memory  of  the  presented  information  (e.g.,  Jacoby  &  Dallas, 

1 981 ).  Compared  with  novices,  do  experts  use  more  or  different  types  of 
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perceptually-  and/or  conceptually-driven  planning  and  briefing  strategies  (and 
what  are  those  strategies)? 

The  results  of  the  CTA  research  will  be  used  to  refine  the  C3PI  design  and  add  to  its 
training  content.  Specifically,  each  experience-related  process  difference  identified 
would  be  assessed  to  identify  an  appropriate  training  strategy,  which  could  then  be 
incorporated  into  C3PI.  In  some  cases,  a  difference  could  be  addressed  by  inserting 
new  types  of  what-if  events  or  what-if  challenges  into  C3PI  databases.  In  other  cases, 
new  planning  tools  might  be  implemented  (e.g.,  to  aid  in  the  use  of  certain  perceptually- 
or  conceptually-driven  planning  strategies)  or  attentional  advice  might  be  added.  In  still 
other  cases,  knowledge  categories  and  knowledge  use  prompts  may  be  refi  ned  or 
expert  advice  video  clips  may  be  added  (e.g.,  to  describe  using  analogies). 

Phase  II  Research  Plans 


To  explore  the  above  hypotheses  and  investigate  the  cognitive  elements  of 
planning  expertise,  we  have  proposed  to  conduct  CTA-based  research  during  Phase  II, 
as  noted  above.  The  focus  of  this  CTA  research  will  be  on  the  planning  processes 
associated  with  what-if  and  contingency  planning.  We  anticipate  that  the  data  and 
results  obtained  will  indirectly  influence  and  directly  feed  into  C3PI  content  and  design. 
The  research  objectives  are  to: 

?  characterize  what-if  and  contingency  planning  processes  in  terms  of  their 
cognitive  elements; 

?  identify  experience-related  differences  in  what-if  and  contingency  planning 
processes  and  their  cognitive  elements;  and 

?  identify  a  subset  of  mission  preparation  measures  based  on  the  identified 
experience-related  differences. 

During  Phase  I,  we  utilized  a  CTA  data  collection  protocol  that  was  designed  to 
link  mission  event  successes  and  difficulties  to  mission  preparation  processes.  This 
protocol,  described  in  the  preceding  section,  provided  insight  into  high-level  mission 
preparation  strategies.  However,  the  protocol  is  not  suited  to  revealing  the  more 
detailed  information  about  planning  processes  and  experience-related  differences  that 
we  now  seek  and  plan  to  use  to  add  to  the  C3PI  training  capabilities. 

Consequently,  we  are  proposing  to  use  a  different  data  collection  strategy  during 
the  Phase  II  research  effort.  Specifically,  we  hope  to  observe  tactical  team  planning 
sessions  (preferably  air  combat  team  planning  sessions)  and  videotape  the  what-if  and 
contingency  planning  portions  of  each  session  (we  would  videotape  the  entire  session 
but  only  use  the  what-if  and  contingency  planning  portions).  As  soon  after  the  planning 
session  as  feasible,  we  would  play  back  those  portions  of  the  planning  session  with  the 
team  lead  while  asking  him  to  elaborate  on  what  he  is  doing  in  the  video,  why,  and  how. 
His  descriptions  of  the  planning  activities  would  be  audiotaped.  We  proposed  to 
conduct  this  data  collection  using  six  planning  teams  with  team  leads  of  varying  levels 
of  experience. 
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Collecting  planning  process  data  is  critical  to  meeting  the  objectives  of  this  CTA. 
However,  if  the  data  collection  plan  just  described  is  found  to  not  be  feasible,  an 
alternative  plan  will  be  developed  based  on  the  data  collection  constraints.  One  such 
alternative  is  to  collect  data  from  teams  in  a  different  mission  planning  domain  that  can 
support  the  resource  requirements  we  describe  (i.e.,  teams  that  are  not  in  the  air 
combat  community).  Another  is  to  arrange  for  AFRL  SMEs  to  hold  mock  planning 
sessions.  Alternative  data  collection  plans  that  require  fewer  resources  include 
collecting  only  the  planning  videotape  data  and  not  the  follow-on  interview  data  and 
decreasing  the  number  of  participating  teams.  Each  of  the  identified  alternatives  is 
associated  with  significant  trade-offs  that  will  affect  the  quality  and  reliability  of  the  CTA 
results.  Consequently,  finding  air  combat  teams  who  will  accommodate  our  proposed 
data  collection  plan  will  be  given  high  priority  and  immediate  attention  at  the  outset  of 
the  proposed  Phase  II  effort. 

Although  data  from  additional  teams  would  improve  the  reliability  of  the  research 
findings,  they  would  also  increase  both  the  risk  and  cost  associated  with  the  research. 
CTAs  can  be  very  time  consuming,  especially  when  they  are  focused  on  a  low  level  of 
analysis,  as  in  this  case.  Furthermore,  it  can  take  as  much  as  a  full  working  day  for  an 
expert  typist  to  transcribe  1  hr  of  an  interview  (Hoffman,  1 987),  and  we  will  most  likely 
not  be  able  to  outsource  the  data  transcribing  to  a  dedicated  transcriptionist  because  of 
its  classified  nature.  Hence,  we  have  chosen  six  as  the  number  of  teams  we  propose  to 
study  because  it  represents  a  number  that  is,  based  on  our  past  CTA  experience,  large 
enough  to  support  the  types  of  analyses  we  plan  to  conduct  yet  small  enough  to  be 
completed  in  a  reasonable  period  of  time  (estimated  timeframe  is  8-9  mths). 

The  CTA  data  analysis  phase  will  involve  transcribing  the  videotaped  and 
audiotaped  data  and  parsing  the  transcriptions  into  tables  of  data  elements,  each  data 
element  representing  a  single  concept  or  piece  of  knowledge.  Audiotape  data  will  be 
represented  in  the  same  table  as  the  videotape  data  for  a  given  team.  Because  the 
audiotape  data  is  supplementary  to  the  videotape  data,  the  data  elements  from  these 
two  sources  will  be  represented  in  parallel  columns  so  they  may  be  considered  together 
across  the  planning  timeline. 

Data  for  all  six  interviews  will  first  be  reviewed  to  gain  insight  into  patterns  and 
trends  across  the  planning  process.  Based  on  this  insight  and  the  results  of  the  Phase  I 
research,  hypotheses  about  the  cognitive  principles  of  planning  and  experience-related 
differences  in  planning  principles  (e.g.,  those  described  in  the  preceding  section)  will  be 
specified.  To  facilitate  the  analysis,  analysis  guidelines  for  evaluating  each  hypothesis 
will  be  developed.  These  guidelines  will  provide  guidance  for  identifying  data  elements 
that  contain  evidence  relevant  to  each  hypothesized  cognitive  principle  and  experience- 
related  difference.  The  data  elements  of  more  and  less  experienced  teams  and  team 
leads  for  a  given  hypothesis  will  be  compared  qualitatively  and  quantitatively,  in  ways 
specified  by  the  analysis  guidelines  (i.e.,  based  on  the  affordances  of  the  data). 
Qualitative  assessments  might  include  assessing  the  data  to  identify  uses  of  analogy. 
These  assessments  might  be  complemented  by  follow-on  quantitative  analyses  of  the 
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number  of  analogies  used  in  each  planning  phase  and  follow-on  qualitative  analyses  of 
the  planner’s  goal  in  using  each  analogy,  as  an  example. 

FUTURE  WORK 

The  Phase  I  work  described  above  laid  the  groundwork  for  our  Phase  II  effort, 
which  will  consist  of  conducting  CTA-based  research  to  investigate  the  cognitive 
elements  of  planning  expertise,  refining  and  adding  to  the  C3PI  design  and  content 
based  on  the  findings  of  that  research,  implementing  the  design,  and  evaluating  the 
value  of  the  system,  both  in  terms  of  perceived  value  by  the  user  and  more  objective 
measures  of  performance  and  expertise  enhancement.  Specific  objectives  proposed  for 
the  Phase  II  work  include  the  following: 

■  Incorporate  cognitive  principles  of  planning  expertise  into  C3PI  design  and  content 

■  Develop  an  intuitive  user  interface  that  supports  training  objectives 

■  Develop  dual-use  C3PI  planning  support  and  training  tools 

■  Develop  mission  brief  and  after  action  review  (AAR)  support  capabilities 

■  Integrate  system  components  to  produce  a  complete  C3PI  prototype 

■  Develop  and  evaluate  the  capability  for  C3PI  to  be  used  collaboratively  by 
distributed  teams 

To  summarize,  the  goal  of  this  effort  is  to  develop  training  for  a  critical  phase  of 
tactical  mission  execution  that,  to  date,  has  been  relatively  neglected  -  mission 
planning.  Further,  this  training  will  focus  on  the  acquisition  of  mission  planning 
expertise  that  facilitates  team  adaptability  in  a  dynamic  environment.  This  training  will 
be  provided  on  a  training  system  (C3PI)  that  must  be:  (1)  compatible  with  the  DMT 
system;  (2)  easy  for  users  to  use  in  training  and  ultimately  in  real  world  planning 
environments;  (3)  robust;  (4)  designed  based  on  sound  research;  and  (5)  found  to 
improve  measures  of  decision  making  and  team  coordination  during  unanticipated 
mission  events. 


32 


REFERENCES 


Bergondy,  M.,  Fowlkes,  J.,  Gualtieri,  J.,  &  Salas,  E.  (1998).  Key  team  competencies  for 
Navy  air  wings:  A  case  study.  Proceedings  of  the  19th  Interservice/  Industry 
Training  Systems  and  Education  Conference.  Arlington,  VA:  National  Training 
Systems  Association. 

Blickensderfer,  E.,  Cannon-Bowers,  J.A.,  &  Salas,  E.  (1997).  Theoretical  bases  for  team 
self-correction:  Fostering  shared  mental  models.  Advances  in  Interdisciplinary 
Studies  of  Work  Teams ,  4,  249-279. 

Cannon-Bowers,  J.  A.,  Rhodenizer,  L.,  Salas,  E.,  &  Bowers,  C.  (1998).  A  framework  for 
understanding  pre-practice  conditions  and  their  impact  on  learning.  Personnel 
Psychology,  51,  291-317. 

Cannon-Bowers,  J.A.,  Tannenbaum,  S.I.,  Salas,  E.,  &  Volpe,  E.  (1995).  Defining 
competencies  and  establishing  team  training  requirements.  In  R.  Guzzo  &  E. 
Salas  (Eds.),  Team  Effectiveness  and  Decision  Making  in  Organizations  (pp. 
333-380).  San  Francisco,  CA:  Jossey-Bass. 

Chase,  W.G.  &  Simon,  H.A.  (1973).  Perception  in  chess.  Cognitive  Psychology,  4,  55- 
SI. 

Chen,  Z.  (2002).  Analogical  problem  solving.  Journal  of  Experimental  Psychology 
Learning,  Memory,  and  Cognition,  28,  81-98. 

Chi,  M.T.H.,  Feltovich,  P.J.,  &  Glaser,  P.  (1981).  Categorization  and  representation  of 
physics  problems  by  experts  and  novices.  Cognitive  Science,  5,  121-152. 

Cohen,  M.S.,  Freeman,  J.T.,  &  Thompson,  B.  (1998).  Critical  thinking  skills  in  tactical 
decision  making:  A  model  and  a  training  strategy.  In  J.  A.  Cannon-Bowers  &  E. 
Salas  (Eds.),  Making  Decisions  Under  Stress:  Implications  for  Individual  and 
Team  Training  { pp.  155-189).  Washington,  DC:  APA. 

Cohen,  M.S.,  Thompson,  B.B.,  Adelman,  L.,  Bresnick,  T.A.,  &  Tolcott,  M.A.,  &  Freeman, 
J.T.  (1995).  Rapid  capturing  of  battlefield  mental  models  (Technical  Report  95- 
3).  Ft  Leavenworth,  Kansan:  Ft  Leavenworth  Field  unit,  US  Army  Research 
Institute. 

Colegrove,  C.M.,  &  Alliger,  G.M.  (2001).  Mission  essential  competencies:  defining 
combat  mission  readiness  in  a  novel  way.  Paper  p  resented  at  the  SAS-038 
NATO  Working  Group  Meeting,  Brussels,  Belgium. 

Cooke,  N.J.,  Cannon-Bowers,  J.A.,  Kiekel,  P.A.,  Rivera,  K.,  Stout,  R.J.,  &  Salas,  E. 
(2000).  Improving  teams’  interpositional  knowledge  through  cross  training. 
Proceedings  of  the  IEA/HFES  2000  Congress  (pp.  2-390  -  2-394).  Santa 
Monica,  CA:  Human  Factors  and  Ergonomics  Society. 

Crane,  P.  (1 999).  Designing  training  scenarios  for  distributed  mission  training. 

Presented  at  the  10th  International  Symposium  on  Aviation  Psychology, 
Columbus,  OH. 


33 


Doane,  S.M.,  Alderton,  D.L.,  Sohn,  Y.W.,  &  Pelligrino,  J.W.  (1996).  Acquisition  and 

transfer  of  skilled  performance:  Are  visual  discrimination  skills  stimulus  specific? 
Journal  of  Experimental  Psychology:  Human  Perception  and  Performance,  22, 
1218-1248. 

Doane,  S.M.,  Pelligrino,  J.W.,  &  Klatsky,  R.L.  (1 990).  Expertise  in  a  computer  operating 
system:  Conceptualization  and  performance.  Human-Computer  Interaction,  5, 
267-304. 

Ericsson,  K.A.  &  Lehman,  A.C.  (1996).  Expert  and  exceptional  performance:  Evidence 
of  maximal  adaptation  to  task  constraints.  Annual  Review  of  Psychology,  47, 
273-305. 

Fowlkes,  J.  E.,  Baker,  D.,  Salas,  E.,  Cannon-Bowers,  J.  A.,  &  Stout,  R.  J.  (2000).  The 
utility  of  event-based  knowledge  elicitation.  Human  Factors,  42,  24-35. 

Fowlkes,  J.,  Dwyer,  D.J.,  Oser,  R.L.,  &  Salas,  E.  (1998).  Event-based  approach  to 
training  (EBAT).  International  Journal  of  Aviation  Psychology,  8,  209-221 . 

Gentner,  D.  (1983).  Structure-mapping:  A  theoretical  framework  for  analogy.  Cognitive 
Science,  7, 155-170. 

Hayes-Roth,  B.  &  Hayes-Roth,  F.  (1979).  A  cognitive  model  of  planning.  Cognitive 
Science,  3,  275-310. 

Hoffman,  P.J.  (1987).  The  problem  of  extracting  the  knowledge  of  experts  from  the 
perspective  of  experimental  psychology.  The  Al  Magazine,  8^53-66. 

Jacoby,  L.L.  &  Dallas,  M.  (1981).  On  the  relationship  between  autobiographical  memory 
and  perceptual  learning.  Journal  of  Experimental  Psychology:  General,  110,  306- 
340. 

Klein,  G.,  &  Miller,  T.E.  (1999).  Distributed  planning  teams.  International  Journal  of 
Cognitive  Ergonomics,  3,  203-222. 

Klein,  G.  &  Pierce,  L.G.  (2001).  Adaptive  teams.  In  Proceedings  of  the  6th  ICCRTS 

Collaboration  in  the  Information  Age  Track  4:  C2  Decision-Making  and  Cognitive 
Analysis.  Retrieved  from:  http://www.dodccrp.org/6thlCCRTS/. 

Klein,  G.,  Pliske,  R.M.,  Crandall,  B.,  &  Woods,  D.  (1999).  Features  of  problem 

detection.  In  Proceedings  of  the  Human  Factors  and  Ergonomics  Society  43rd 
Annual  Meeting  (pp.  133-137).  Santa  Clara,  CA:  HFES. 

Kozlowski,  S.W.J.  (1998).  Training  and  developing  adaptive  teams:  Theory,  principles, 
and  research.  In  J.  A.  Cannon-Bowers  E.  Salas,  (Eds.).  Making  Decisions 
Under  Stress:  Implications  for  Individual  and  Team  Training  (pp.  115-1 53). 
Washington,  DC:  APA. 

MacMillan,  J.  Entin,  E.B.,  &  Serfaty,  D.  (1993).  Defining  and  assessing  expertise  in  the 
complex  domain-measures  based  on  theory.  Paper  presented  at  the  37th  Annual 
Meeting  of  the  Human  Factors  and  Ergonomics  Society,  Seattle,  WA. 


34 


Neville,  K.,  Fowlkes,  J.E.,  Walwanis,  M.M.,  Bergondy,  M.L.,  &  Strini,  T.  (June  2002). 
Characteristics  of  team  coordination  expertise  in  a  large,  distributed  tactical 
team:  A  cognitive  task  analysis  of  naval  air  wing  strike  mission  execution. 
Manuscript  submitted  to  NAVAIR  Orlando  -  Training  Systems  Division  for  review. 

Nullmeyer,  R.T.,  Crane,  P.,  Cicero,  G.,  &  Spiker,  V.A.  (2000).  A  bridge  between 

cockpit/crew  resource  management  And  distributed  mission  training  for  fighter 
pilots.  Proceedings  of  the  20th  Interservice/Industry  Training  Systems  and 
Education  Conference.  Orlando,  FL. 

Pierce,  L.  &  Pomranky,  R.  (2001).  The  Chameleon  project  for  adaptable 

commanders  and  teams.  Proceedings  of  the  Human  Factors  and  Ergonomics 
Society  45th  Annual  meeting  (pp.  513-51 7).  Santa  Monica,  CA:  Human  Factors 
and  Ergonomics  Society. 

Rosenberger,  J.D.  (1 995).  The  burden  our  soldiers  bear:  Observations  of  a  senior 

trainer  (O/C).  Combat  Training  Center  Quarterly  Bulletin.  Retrieved  28  January 
2002  from  http://call.army. mil. products/ctc_bull. 95-1 1/ctcl  -01  .htm. 

Ross,  K.G.  &  Pierce,  L.G.  (2000).  Cognitive  engineering  of  training  for  adaptive 

battlefield  thinking.  In  Proceedings  of  IEA  14th  Triennial  Congress  and  HFES 
44th  Annual  Meeting  {pp.  410-413).  Santa  Monica,  CA:  Human  Factors  and 
Ergonomics  Society. 

Sidor,  G.  J.  (2002).  Distributed  Mission  Training  collaborative  planning, 

briefing,  and  debriefing  system.  Proceedings  of  the  3rd  International 
Symposium  on  Collaborative  Technologies  and  Systems,  264-267. 

Smith,  E.M.,  Ford,  J.K.,  &  Kozlowski,  S.W.J.  (1997).  Building  adaptive  expertise: 

Implications  for  training  design.  In  M.A.  Quinones  &  A.  Dudda  (Eds.),  Training 
fora  Rapidly  Changing  Workplace:  Applications  for  Psychological  Research  (pp. 
89-118).  Washington,  DC:  APA. 

Spiker,  A.  &  Nullmeyer,  R.  (1 995).  Measuring  the  effectiveness  of  mission  preparation  in 
the  Special  Operations  Forces  (AL/HR-TR-1 995-007).  Mesa,  AZ:  Armstrong 
Laboratory. 

Spiker,  A.,  Nullmeyer,  R.  &  Tourville,  S.  (2001).  Relationship  between  mission 

preparation,  and  performance  during  combat  mission  training.  Proceedings  of  the 
22nd  Interservice/  Industry  Training  Systems  and  Education  Conference. 
Arlington,  VA:  National  Training  Systems  Association. 

Spiker,  A.,  Nullmeyer,  R.,  Tourville,  S.  &  Silverman,  D.  (1997).  Effects  of  mission 

preparation  on  crew  combat  mission  performance.  Proceedings  of  the  Human 
Factors  and  Ergonomics  Society  41st  Annual  Meeting.  Santa  Clara,  CA:  HFES. 

Spiker,  V.A.,  Tourville,  S.J.,  Bragger,  J.,  Dowdy,  D.  &  Nullmeyer,  R.T.  (1999). 

Measuring  C-5  crew  coordination  proficiency  in  an  operational  win.  Proceeding  of 
the  20th  Interservice/  Industry  Training  Systems  and  Education  Conference. 
Arlington,  VA:  National  Training  Systems  Association 


35 


Spiro,  R.J.,  Feltovich,  P.J.,  Jacobson,  M.J.,  &  Coulson,  R.L.  (1992).  Cognitive  flexibility, 
constructivism,  and  hypertext:  Random  access  instruction  for  advanced 
knowledge  acquisition  in  ill-structured  domains.  Found  in  T.M.  Duffy,  D.H. 
Jonassen  (Eds.),  Constructivism  and  the  Technology  of  Instruction:  A 
Conversation  (pp.  57-75).  Hillsdale,  NJ:  Erlbaum. 

Stout,  R.J.  (1995).  Planning  effects  on  communication  strategies:  A  shared  mental 
model  perspective.  In  Proceedings  of  the  39th  Annual  Meeting  of  the  Human 
Factors  and  Ergonomics  Society  (pp.1 278-1 282).  Santa  Clara,  CA:  HFES. 

Stout,  R.J.,  Cannon-Bowers,  J.A.,  &  Salas,  E.  (1997).  A  team  perspective  on  situational 
awareness  (SA):  Cuing  training.  In  Proceedings  of  the  1997  Interservice/Industry 
Training,  Simulation,  and  Education  Conference.  Arlington,  VA:  National  Training 
Systems  Association. 

Stout,  R.J.,  Cannon-Bowers,  J.A.,  Salas,  E.,  &  Milanovich,  D.M.  (1999).  Planning, 
shared  mental  models,  and  coordinated  performance:  An  empirical  link  is 
established.  Human  Factors,  41,  61  -71 . 


36 


APPENDIX  A 


C3PI  Event-Based  Interview  Protocol 

Introduction  (This  is  read  to  participant  at  beginning  of  interview) 

We  are  conducting  these  interviews  for  an  SBIR  project  aimed  at  enhancing  DMT- 
based  training  to  build  expertise  in  planning.  Through  the  interviews,  we  are  trying  to 
understand 

What  aspects  of  planning  contribute  to  mission  performance? 

What  aspects  of  planning  contribute  to  replanning  capability? 

We  are  going  to  ask  you  about  a  series  of  events  or  problems  that  might  occur  during 
the  conduct  of  a  defensive  counter  air  (DCA)  lane  defense  mission  flown  by  an  F-16  4- 
ship.  Our  main  interest  is  in  how  you  would  deal  with  each  eve  nt  and  how  the  mission 
plan  or  the  mission  planning  process  would  help  prepare  you  to  deal  with  each  event. 
With  respect  to  the  mission  plan  and  the  planning  process,  we  would  like  to  hear  about 
things  experts  do  that  would  help  the  4-ship  deal  effectively  with  a  problem  and  about 
things  less  experienced  people  may  do  that  may  decrease  the  team’s  chances  for 
success. 

We  would  like  to  audio  tape  this  interview.  Also,  we  would  like  to  begin  with  the 
assumption  that  it  can  be  kept  at  an  unclassified  level.  However,  if  you  say  something 
that  is  classified  or  think  we  should  know  about  something  that  involves  classified 
information,  please  let  us  know.  We  have  permission  to  collect,  store,  and  use 
information  that  is  classified  up  to  the  Secret  level. 

To  reiterate  our  objectives,  we  are  going  to  ask  you  how  you  would  deal  with  each  of  a 
series  of  events  and  how  the  mission  plan  or  the  mission  planning  process  would  help 
prepare  you  for  dealing  with  each  event.  With  respect  to  the  mission  plan  and  the 
planning  process,  we  would  like  to  hear  about  things  experts  do  that  would  help  the  4- 
ship  deal  effectively  with  a  problem  and  about  things  less  experienced  people  may  do 
that  may  decrease  the  team’s  chances  for  success. 

If  you  can  think  of  specific  examples  of  situations  you  have  been  in  or  heard  about  that 
are  relevant  to  the  questions,  those  would  be  especially  valuable  and  helpful  for  us  to 
hear. 

Do  you  have  any  questions  for  us  before  we  begin? 

Events  Posed  (each  event  is  followed  by  questions  listed  below) 

1 .  During  the  CAP  phase,  you  find  out  that  #2  and  #3  have  an  alibi  -  neither  has 
working  radar. 

2.  During  the  merge  preparation  phase,  #3  makes  confusing  sorting  comms  as  you  sort 
against  4  hostile  aircraft  and  you  are  not  sure  if  he  is  targeting  the  one  you  wanted 
him  to 
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3.  Hostile  a/c  have  been  trying  to  engage  you  in  cat  and  mouse  game.  Fighters  in  the 
“next  lane”  have  fallen  out  so  now  you  have  to  cover  their  lane. 

4.  Hostile  a/c  have  been  trying  to  engage  you  in  cat  and  mouse  game.  Fighters  in  the 
“next  lane”  have  committed  on  a  group.  You  lose  AWACS  (AWACS  goes  midnight). 

5.  You’ve  committed  two  ships  to  what  you  thought  was  a  two  ship,  but  breaks  out  to 
be  a  four  ship 

6.  During  the  commit  to  the  4-ship,  you  stop  hearing  comms  (or  comms  are  degrading) 
and  your  SA  drops. 

7.  During  the  merge  preparation  phase,  #3  is  inexperienced  and  begins  to  panic  as  you 
approach  VID  situation 

8.  During  the  merge  phase,  #2  breaks  from  your  gameplan  and  targets  hostile  aircraft 
before  getting  into  the  correct  position/location 

9.  During  the  merge  phase,  #3  breaks  from  your  gameplan  by  going  too  far  forward  to 
target  the  threats  instead  of  turning  around  at  range  (I  think  this  is  the  same  as  one 
you  give  below)  You  are  in  a  grinder.  The  plan  was  to  leave  at  range.  As  you  are 
wheeling  around  (to  head  back  toward  the  threat),  you  see  your  friendlies  mixed  in 
with  the  threats  in  front  of  you,  and  you  are  not  hearing  comm. 

1 0.  Post  merge,  hostiles  did  not  blow  up  like  they  were  supposed  to.  You’re  “running 
away”  and  trying  to  build  a  mental  picture. 

Questions  Asked  After  Description  of  Each  Event  Listed  Above 

1 .  Have  you  been  in  this  situation  before? 

2.  What  would  be  one  of  the  first  things  you  would  expect  to  hear?  Why? 

3.  How  could/did  the  plan  or  planning  process  contribute  to  successfully  detecting  this 
situation? 

4.  What  would  be  an  example  of  excellent  comm  in  this  situation  by  you,  by  your 
wingman,  by  AWACS? 

5.  How  does  the  plan  and  planning  process  help  prepare  you  for  this  type  of  effective 
comm? 

6.  What  decisions  would/did  you  make?  What  are  the  considerations? 

7.  How  would/did  the  plan  or  planning  process  contribute  to  your  decision? 

8.  What  could/did  go  wrong  in  the  team’s  reaction  to  this  situation?  What  mistakes 
did/might  the  team  or  a  team  member  make  in  dealing  with  this  situation  (if  the  re 
are/were  are  many  things,  pick  2  or  3)? 

9.  How  could/did  the  plan  or  planning  process  help  prevent  these  things  from  going 
wrong? 

In  what  other  ways  could/did  the  plan  or  planning  process  contribute  to  successfully 

responding  to  this  situation  as  a  team?  Interviewer:  listen  for  knowledge  of  constraints, 

flow/timing,  teammates,  own  abilities  and  limitations,  dynamic  threat  situation. 

1 0.  Can  you  think  of  2  or  3  examples  of  shortcomings  of  a  plan  or  planning  process  that 
would/did  decrease  the  team’s  chances  for  successfully  reacting  to  this  situation? 

1 1 .  Under  what  conditions  might  these  shortcomings  in  a  plan  or  planning  process 
occur?  Why  might  they  occur? 

12.  What  are  your  assets  for  building  SA  and  how  would  you  use  them  now?  How  does 
or  should  the  plan  help  prepare  you  for  this? 
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APPENDIX  B 


Event-Based  Interview  Data  Syntheses  for  Two  Air  Combat  SMEs 

Notes.  Knowledge  category  codes  are  defined  in  Table  3.  Interview  content 
represented  in  bold  font  are  interviewer  verbalizations. 

SME  1 


Description  of 
Interview  Content 

Knowledge 

Categories 

Interview  Content 

What-lf  Examples 

Wide  Azimuth-group 
out  of  lane 

SA 

And  you  kind  of  brief  in  the  scenario  is,  "ok,  you've  got  a  4- 
ship  of  F-15s  up  here  in  the  northern  lane,  a  4-ship  of  F-18s  in 
the  southern  lane".  Are  they  really  out  there,  no,  they're  virtual 
entities  in  the  sky.  So  what  we'll  do,  we'll  give  them  a  very 
wide  Azimuth  that's  38  miles  apart  and  what  we're  trying  to  do 
is  see  if  he  even  recognizes  that  "hey,  one  of  these  groups  is 
out  of  my  lane". 

Faulty  radar 

mi 

get  on  CAP  and  somebody's  radar  isn't  working 

Sorting  comms 

DPC 

you're  going  in  to  the  merge  or  sorting  and  your  #3  doesn't 
say  something,  or  doesn't  reply  to  your  sorting  comms,  do  3 
and  4  sort  together  or  does  it  depend  on  how  many  fighters 
they're  coming  up  against. 

Next  lane  falls  out 

DPC 

bandits  are  giving  you  a  hard  time  and  fighters  in  the  next 
lane  fall  out,  would  you  cover  their  lane 

Style  out  of  range 

DPC,  SA 

If  there  are  two  styles  and  one  comes  up  north  that's  out  of 
your  30  mile  range,  is  that  still  something  you  would  take  on 

Two  fighters  have  two 
packages 

DPC,  SA 

What  are  you  going  to  do  if  you've  got  1  &2  and  you've  got  2 
packages  in  the  south,  can  one  take  one  package  and  2  the 
other,  or  do  you  just  see  which  is  closer 

Targeting  same 
package 

SA,  DPC 

Have  you  ever  been  in  a  situation  where  you've  been  in  a  4 
ship  and  are  targeting  a  hostile  group  and  come  to  find  out 
that  somebody  from  some  other  group  is  targeting  the  same 
package 

Same  area  as  another 
4-ship 

SA 

Have  you  had  it  where  you're  operating  in  the  same  area  as 
another  4-ship  from  another  lane... 

Blue  on  blue 

TF 

A  good  blue  on  blue  example... because  that  really  relates  to 
knowing  where  people  are  and  where  you  need  to  flow 
coming  out  of  engagements... 

Plan  Elements 

What-ifs 

DPC,  SA 

Like  1  told  the  other  groups  when  they  ask  me,  it's  the  groups 
that  think  about  the  what-ifs,  if  you  bring  those  out  in  the 
mission  planning  and  preparation  it  really  helps. 

Commander's  Intent 

DPC 

if  you  say  "think  about  your  mission,  think  about  your  risk 
level,  think  about  what  it  affects"  in  that  context  1  think  that'll 
help  a  lot. 

Bounding 

range/package  criteria 

MEE 

My  lane  is  30  miles  wide,  so  in  my  brief  and  in  your  planning, 
you  should  have  a  bounding  range  or  package  criteria. 

Competencies 

K4 

You've  got  to  focus  on  your  primaries  and  you  actually  get 
better  when  you  get  into  the  supporting  competencies, 
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knowledge  and  skills. 


Contingency  Planning 

DPC 

It's  all  contingency  planning 

4-ship  involvement 

K4 

The  entire  4-ship  should  be  part  of  the  mission  plan,  should 
be  part  of  the  brief 

Priorities  in  comms 

MS 

But  if  you  really  illustrate,  "guys,  if  no  one's  defensive,  threat 
or  spike  and  you  hear  someone  call  blind..."  you  really  go 
through  those  situations  and  you  point  it  out. 

Bullseye,  enemy 
tactics 

ET,  MEE 

If  they  don't  talk  about  it  and  the  flight  lead  doesn't  get  up  and 
say  "hey  guys  there's  an  SA-2,  SA-6,  SA-8  blah  blah  blah, 
guess  what,  we  don't  want  to  be  anywhere  15  miles  closer  to 
bullseye.  We  don't  need  to  be  there,  so  don't  let  them  lure  us 
in". 

Enemy  tactics 

ET,  K4,  DPC 

So  mission  planning  and  you  see  groups  go  out  there  going 
and  killing  everything  in  front  of  us  and  they  just  trundle  in  and 
the  red  air  doesn't  get  them,  the  SAMs  do.  Or  vice  versa, 
they'll  start  paying  such  attention  to  the  SAMs,  that  the  red  air 
gets  them.  They  didn't  mission  plan,  they  didn't  think  about  it, 
they  didn't  say  that's  not  their  mission  to  go  and  do  that  and  1 
think  you'll  see  that  a  lot. 

Plan  and  Brief  Organization 

What-ifs,  no  missiles, 
no  radar 

DPC,  MS 

Usually  guys  reserve  this  for  the  end  of  the  brief,  let's  say 
things  start  going  wrong,  we're  down  to  one  person  in  the 
element  has  no  missiles,  very  similar  to  someone  having  no 
radar  and  then  build  a  plan  from  right  there. 

Cold  ops  priorities 

TF,  SA, 

and  the  best  ones  I've  seen  is  in  that  Cold  Ops  or  something 
like  this,  a  guy  spent  a  good  chunk  of  the  brief,  talking  about 
that.  And  it  was  almost  a  laundry  list;  this  happens  first,  then 
this,  we  all  get  together,  once  we're  all  together,  now  1  say 

GCI  picture.  Everybody  shuts  up,  GCI  paints  the  picture,  if  1 
don't  understand  it,  I'll  ask  a  question  or  I'll  call  out  from  Viper 

1  to  the  nearest  group... and  you've  heard  all  these  things 
talked  about  and  said,  now  everybody's  got  the  picture,  here's 
the  game  plan,  Viper  3,  you'll  pitch  back  first,  target  blah  blah 
blah,  any  questions,  silence,  execute. 

Variations  in  Planning  and  Briefing 

What-ifs 

PE,  DPC 

I've  worked  w/the  other  groups  that  have  come  through  the 
mission  planning  cell  and  it's  the  groups  that  talk  about  these 
type  of  things  that  go  wrong  that  do  the  best.  And  they  have  a 
plan. 

Commander's  Intent, 
background  info 

MEE 

I'll  start  out  the  brief  and  give  a  big  overview  of  what  we're 
going  to  do  and  maybe  the  intel  officer  will  come  up  and  give 
the  intel  report  for  the  day.  And  I'll  go  "ok  guys,  the  big  picture 
today,  we  are  DCA  lane  defense,  we're  defending  this  lane, 
here's  the  parameters,  here's  what  we're  due,  here's  our 
weapons  load -out,  here's  what  we're  expecting  to  see"  and 
everything  like  that.  And  now  I'll  grab  the  contracts  card  and 

I'll  go... because  most  of  this  stuff  is  standard. 

Lead  techniques 

N/A 

There's  two  different  schools  of  thought  by  they  way,  that's  all 
briefing  techniques.  Some  people  like  to  get  up  there  and  just 
say,  "1  am  king,  listen  to  me  preach"  other  people  like  to  say, 

"3  what  are  we  going  to  do,  or  GCI,  call  this  picture  out" 
practice  and  get  them  involved  right  then. 

Presentation  strategies 

N/A 

It's  techniques  there,  it's  however  you  want  to  do  it.  You're 
talking  to  a  map,  or  at  least  draw  it  up  on  the  whiteboard  and 
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they'll  talk  to  it  when  they  have  Rodan  in  the  north  and  this 
person  in  the  south. _ 


Flow  of  elements 

TF,  DPC 

If  you  get  a  good  lead  that  briefs  and  knows  the  mission  and 
knows  the  plan  and  talks  about  where  the  other  blue  groups 
are,  the  other  folks,  that  usually  eliminates  that,  still  happens, 
and  then  you  also  go  into  the  what-if  stuff  goes  wrong  and 
we're  separated,  how  do  we  fix  that. 

Cold  ops,  post-merge 

DPC,  MS 

Bringing  it  back  to  our  mission  and  what  we're  trying  to  do.  4-1 
don't  want  you  running  west  bound  if  we're  DCA  lane  defense 
in  the  (?)  war,  you  do  me  no  good  out  there  alone,  so  as  long 
as  you're  survived,  and  if  1  illustrate  this,  now  1  avoid  the  blue 
on  blue  and  1  get  people  thinking  about  that,  as  long  as  you're 
surviving  and  you're  not  spiked  anymore,  start  heading  back 
west  and  you'll  even  hear  guys  brief  for  the  cap  plan  "ok, 
you're  alone  and  afraid  and  everything's  going  bad.  If  you're 
defensive,  run  back  to  the  cap,  first  one  there  mans  the  cap, 
set  up  cap  there  and  someone  will  come  back  and  find  you,  or 
we'll  all  regroup". 

Leadership 

Lead  characteristics 

PE 

The  lead  makes  a  huge  difference  because  it's  going  to  come 
across  in  the  briefing,  and  you’ve  probably  sat  through  some 
and  witnessed  enough  to  know  a  good  one  from  a  bad  one. 
Just  from  body  language,  affirmation  in  what  you're  saying 
and  what  everyone's  going  to  do,  and  then  during  the 
execution  of  the  flight,  it's  that  flight  lead  that's  been  in  that 
situation  before  that  his  photo  album  is  full  and  his  brain. ..or 
he's  at  least  given  it  enough  thought  to  react  to  it  quickly  and 
that  stems  from  mission  planning  and  briefing. 

Sorting 

K4 

Or  if  in  real  time  1  just  hear  them  say  "4  sort  leaders"  1  say 
"negative,  4  sort  trail".  And  you  step  on  it. 

Intercepts 

SA,  K4 

As  a  leader,  1  need  to  know  where  all  these  folks  are,  1  have 
to  run  my  intercepts  to  these  trailers. 

Lead 

But  I'm  orchestrating  the  whole  thing,  I'm  up  there  the 
conductor,  1  may  ask  the  questions. 

Flow  of  elements 

TF,  SA,  DPC 

If  you  get  a  good  lead  that  briefs  and  knows  the  mission  and 
knows  the  plan  and  talks  about  where  the  other  blue  groups 
are,  the  other  folks,  that  usually  eliminates  that,  still  happens, 
and  then  you  also  go  into  the  what-if  stuff  goes  wrong  and 
we're  separated,  how  do  we  fix  that. 

Planning  Strategies  that  Support  Execution 

No  radar,  no  missiles 

DPC,  MS 

Usually  guys  reserve  this  for  the  end  of  the  brief,  let's  say 
things  start  going  wrong,  we're  down  to  one  person  in  the 
element  has  no  missiles,  very  similar  to  someone  having  no 
radar  and  then  build  a  plan  from  right  there. 

Contingency  planning 

MS,  DPC 

Something  like  where  you're  in  this  big  package  and  you  don't 
have  time  to  say  "sorry  guys,  we're  out",  so  now  you  need  to 
figure  out  who's  going  to  be  the  most  experienced  in  the  flight, 
you  don't  really  want  to  renumber  the  flights,  you  just  have  to 
have  this  briefed 

Contingency  planning 

DPC 

It  is  a  somewhat  standard  plan.  It's  in  most  folks'  brief  to  say 
"you  take  the  tact  lead  of  that  element,  you're  the  primary 
shooter"  whatever  the  case  may  be. 

Sort  plan 

K4 

1  brought  a  brief  in  and  a  contracts  card  that  gets  handed  out. 
You  talk  about  your  sort  plan  and  your  hostile  and  who's  the 

owner  of  the  group,  who's  the  sharer  of  the  group 

Targeting 

K4,  TF 

If  I'm  the  primary  person  targeted  in  that  flight  and  my  entire 
flight  is  going  to  go  tag  it,  it's  my  job  to  build  the  3D  picture  for 
them  so  they  can  revert  back  to  that  knowledge  and  say  "he 
told  me  it  was  a  4-ship  container"  so  1  would  know  as  4  I'd 
want  to  be  on  the  far  trail  or  on  the  leader,  depending  on  what 

1  do. 

Radar  assignments 

K4,  LM 

Everybody's  got  to  know,  if  our  comm  goes  out  or  if  we  get 
comm. -jammed  everybody  should  know  where  their  radar 
should  be  from  mission  planning  and  the  brief. 

GCI  info 

DTC,  SA 

I'll  tell  GCI,  "my  286  brain  can  only  handle  so  much 
information  at  once,  just  tell  me  'groups  marshalling  west  of 
bull  or  2nd  package,  north  of  bull,  bullseye"  and  then  say 
"leading  edge  is  2  groups  azimuth  10,  blah  blah  blah"' 
because  that's  all  1  can  deal  with  now. 

Practice  Comms 

SA,  DTC 

Hey,  it's  an  interactive  brief,  hopefully  "2  you  see  this,  what 
are  you  going  to  say".  Or  more  likely,  in  the  event  of  the 
package  situation  which  is  what  we're  dealing  with  right  now, 
someone  outside  of  our  area  of  responsibility,  "GCI  what  are 
you  going  to  call  this"  and  I'll  get  an  example  of  what  he's 
going  to  say.  If  it's  incorrect,  I'll  correct  him  right  there  on  the 
spot 

Frequencies 

K4,  MEE 

One  thing  that  we  can  bring  out  in  this  thing  with  the  mission 
planning  is  talk  about  the  UHF  frequencies,  the  VHF 
frequencies. 

Missile  management 

LM,  K4 

A  DCA  lane  where  what  we  call  is  WRM  -war  reserve  man 
missiles,  anyway,  your  missile  management,  I've  only  got  a 
certain  number  of  beyond  visual  range  shooting  missiles 
amongst  my  4-ship,  so  1  just  can't  go  out  there  and  the  first 
thing  1  see,  start  wailing  away.  1  have  to  think  about 
conserving  those,  so  yes  if  1  had  a  group  double  targeted,  and 
I've  got  two  people  shooting  two  missiles  each  into  that  group, 
that's  four  missiles  that  may  or  may  not  only  kill  one  or  two 
people.  I've  just  cut  down  my  missiles  by  25%  and  that's  a  big 
part  of  it.  Segue  is,  in  discussing  what  I'm  trying  to  accomplish 
here,  bring  up  what  the  weapons  load  is. 

Comm  priorities 

TF,  K4 

It  kind  of  goes  back  to  comm. -priorities  which  is  mission 
planning  which  is  talking  about  frequencies  and  what  really, 
really  helps,  and  we  get  into  it  a  little  bit  here,  but  is  in  the 
mission  planning,  knowing  where  everybody  else  is. 

Cold  ops 

TF 

Cold  Ops,  what  happens  then,  what  is  my  flow,  what  is  my 
mission  and  that  kind  of  goes  all  the  way  back  to  mission 
planning. 

Flow 

TF 

And  flow,  what  is  my  mission?  Well,  that  should  have  been 
totally  obvious  before  we  even  stepped  in  the  jet. 

Fighter  presentation 

PE,  SA,  K4, 

TF 

I've  seen  groups  go  "let's  grind  today  and  let's  do  this"  and 
they  don't  talk  about  it. ..and  sure  enough,  pitch  back  in, 

"declare  group  blah  blah  blah"  and  one  of  two  things 
happens.  Either  they  waste  all  their  brain  bytes  and  SA  on 
figuring  out  that  that's  1  &2  and  if  1  had  talked  about  it  in  the 
brief  and  mission  planning... 

Team  participation 

PE 

You  kind  of  get  the  feeling  that  the  more  everyone's  involved 
in  the  plan,  obviously  the  better  off  we're  going  to  be. 

Flow  of  strike  package 

TF 

They  say  "hey  we'll  have  Rodan  in  the  north  and  this  in  the 
south  and  20  miles  behind  us  will  be  this  group”  and  if  I’m  the 
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OCA  sweep  I'll  say  "hey  these  guys  are  starting  40  miles 
behind  us,  so  we  have. ..we'll  be  able  to  push  out,  kill 
everything  we  can,  but  if  you  get  in  a  bad  situation,  you  can 
only  abort  once  but  we've  only  got  one  minute  cold  because 

_ strike  package  is  going  to  be  on  our  nose  in  a  minute". _ 

Flow  TF  We  either  have  to  tell  them  to  spin  or  we  need  to  be  aware 

that  those  are  the  good  briefs,  that's  when  everybody's  going 
"we're  in  a  mission  here,  we're  in  a  flow,  we  need  to  know 

_ where  our  flows  are,  what's  going  on". _ 

Communication-cold  SA,  MS  If  you  don't  know  where  you  are  or  what's  going  on,  start 

ops?  heading  back  towards  the  cap,  head  back  towards  the  cap 

and  start  talking  to  someone.  When  they  start  talking, 
everyone  else,  if  you're  not  in  one  of  these  other  priorities 
where  you're  defensive  threat  spiked  or  getting  a  declaration, 
shut  up.  So  the  other  guy  can  talk,  his  element  man  can  find 

_ him,  then  the  whole  4-ship..." _ 

Comm  priority-group  TF,  K4,  SA  So  you  avoid  those  when  the  guys  go  through  these  and  go 

flow  over  comm. -priorities  and  go  "here's  the  situation,  3  &4  get 

separated..."  and  that's  when  you  see  it  happen  a  lot.  3  will 
go  "declare  group  blah  blah"  and  it's  4  because  they  don't 
know,  they  haven't  talked,  they  haven't  allowed  this  to  happen 

_ by  knowing  what  may  happen. _ 

Comm  priority  MS,  K4  But  if  you  really  illustrate,  "guys,  if  no  one's  defensive,  threat 

or  spike  and  you  hear  someone  call  blind..."  you  really  go 

_ through  those  situations  and  you  point  it  out. _ 

Cold  ops  comms  K4,  DPC,  TF  And  the  best  ones  I've  seen  is  in  that  Cold  Ops  or  something 

like  this,  a  guy  spent  a  good  chunk  of  the  brief,  talking  about 
that.  And  it  was  almost  a  laundry  list;  this  happens  first,  then 
this,  we  all  get  together,  once  we're  all  together,  now  I  say 
GCI  picture.  Everybody  shuts  up,  GCI  paints  the  picture,  if  I 
don't  understand  it,  I'll  ask  a  question  or  I'll  call  out  from  Viper 
1  to  the  nearest  group... and  you've  heard  all  these  things 
talked  about  and  said,  now  everybody's  got  the  picture,  here's 
the  game  plan,  Viper  3,  you'll  pitch  back  first,  target  blah  blah 

_ blah,  any  questions,  silence,  execute. _ 

Commander's  Intent  DPC,  TF,  MS  Bringing  it  back  to  our  mission  and  what  we're  trying  to  do.  4-1 
Blue  on  Blue  don't  want  you  running  west  bound  if  we're  DCA  lane  defense 

in  the  (?)  war,  you  do  me  no  good  out  there  alone,  so  as  long 
as  you're  survived,  and  if  I  illustrate  this,  now  I  avoid  the  blue 
on  blue  and  I  get  people  thinking  about  that,  as  long  as  you're 
surviving  and  you're  not  spiked  anymore,  start  heading  back 
west  and  you'll  even  hear  guys  brief  for  the  cap  plan  "ok, 
you're  alone  and  afraid  and  everything's  going  bad.  If  you're 
defensive,  run  back  to  the  cap,  first  one  there  mans  the  cap, 
set  up  cap  there  and  someone  will  come  back  and  find  you,  or 

_ we'll  all  regroup". _ 

Enemy  capabilities  ET,  K4  But  if  we  bring  out... we  do  this  in  our  Intel  scenario,  "you're 

going  to  have  maneuvering  bandits,  they're  AMRAM  aware" 
or  whatever  the  case  may  be,  so  they're  going  to  try  to  defeat 
you. ..so  now  you  need  to  think  about,  we  could  bring  out  "hey 
guys,  think  about  your  missile  WRM,  how  many  missiles  you 

_ have  in  the  flight." _ 

Enemy  capabilities  ET,  TF  If  they  don't  talk  about  it  and  the  flight  lead  doesn't  get  up  and 

say  "hey  guys  there's  an  SA-2,  SA-6,  SA-8  blah  blah  blah, 
_ guess  what,  we  don't  want  to  be  anywhere  15  miles  closer  to 
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bullseye.  We  don't  need  to  be  there,  so  don’t  let  them  lure  us 
in". 


Enemy  threats 

ET,  DPC 

So  mission  planning  and  you  see  groups  go  out  there  going 
and  killing  everything  in  front  of  us  and  they  just  trundle  in  and 
the  red  air  doesn't  get  them,  the  SAMs  do.  Or  vice  versa, 
they'll  start  paying  such  attention  to  the  SAMs,  that  the  red  air 
gets  them.  They  didn't  mission  plan,  they  didn't  think  about  it, 
they  didn't  say  that's  not  their  mission  to  go  and  do  that  and  1 
think  you'll  see  that  a  lot. 

Execution  Challenges 

Contingency  planning 

DPC,  MS 

Now  you're  really  thinking  "we  can  no  longer  be  as  offensive 
minded  as  we  wanted  to  be,  we  need  to  go  out  and  say...#1 
would  be  the  primary  shooter,  and  2  would  just  hang  on  and 
don't  loose  sight.  Now  #3  has  to  kind  of  give  #4  (who's 
probably  the  youngest  guy  on  your  flight  w/the  least  amount 
of  experience)  has  to  be  able  to  say  4,  lock  here  without 
knowing  what's  out  there,  shoot  as  many  as  you  can  and 
we're  going  to  leave  by  a  certain  range". 

Contingency  planning 

DPC,  MS,  LM 

If  1  have  a  really,  really  inexperienced  #4,  now  1  may  do  the  2 
and  4  swap  although  that's  a  lot  more  difficult  logistically  and 
to  make  that  happen  w/out  all  the  confusion  and  now 
renumbering  the  flight  or  something  like  that. 

Lethality,  commander's 
intent 

DPC 

So  now  1  have  to  think,  "oh  my  gosh,  not  only  can  you  not 
shoot  anyone  air-wise,  he  has  to  go  on  very  limited  bomb 
dropping  capability  when  he  gets  to  the  target".  So  is  it  even 
worth  it  to  me  to  bring  him  along.  Or  do  1  just  say  "go  home" 
and  now  I’m  a  3-ship. 

Communications 

K4,  LM 

If  that’s  happening  in  the  air  and  1  recognize  it,  realize 
everybody  sounds  great  outside  of  15  miles,  it's  15  miles  and 
in  when  things  that  start  getting  confusing  and  there  might  not 
be  that  com  time  to  correct  #3. 

Contracts 

K4,  LM 

I'd  hopefully  fix  it  from  my  brief  and  1  have  a  contracts  card 
and  I'm  already  planning  what  I'm  going  to  say  to  3  in  the 
debrief  when  1  get  out  because  he's  obviously  messed  up.  If  1 
have  the  time,  I'm  going  to  correct  him  in  the  air,  has  it 
happened,  yes.  Situationally,  you've  really  described  the 
situation  right  here  and  90%  of  the  time  you  don't  have  time  to 
correct  it,  if  you  even  recognize  it. 

Sorting 

TF,  DPC 

They're  out  there  in  a  supporting  role,  I'm  coming  in  to  ID  this 
group  and  1  hear  3  say  "4  sort  far  trail"  just  as  an  example  or 
"northern  trailer"  or  "southern  trailer"  in  this  case  would  be  a 
better  thing.  Then  1  know  that  3  just  put  4's  radar  right  where 

I’m  going  and  that's  not  good.  "Negative,  4  sort  southern 
region"  in  that  situation. 

Targeting/sorting 

MEE 

As  far  as  targeting  sorting,  that's  where  we  make  our  money, 
so  to  have  a  mistake  on  that  should  be  just  that,  purely  a 
human  error  and  mistake. 

Frequencies 

K4,  DTC 

What  happens  a  lot  of  times  is  people  are  using  different 
radios,  different  radio  freqs,  usually  there's  one  big  package 
strike  frequency,  but  kind  of  the  rules  of  engagement  are  keep 
that  as  clean  as  possible 

Listening  skills 

LM,  K4 

First  thing  that  drops  out  of  everyone  when  you're  flying  out 
here  is  your  listening  skills.  Your  SA  starts  to  go  away,  you 
stop  listening  and  you're  on  transmit  only  and  not  receiving 
any  information. 

SA 

SA,  LM,  K4 

People  get  in  the  same  part  of  the  sky  because  SA  has  gotten 
lower  and  lower  and  lower  or  a  critical  piece  of  information 
was  set  across  the  wrong  radio  frequency. 

Teammate  limitations 

DTC,  LM 

So  they  [GCI]  might  not  even  recognize  that  2  people  are 
going  after  the  same  person.  You  would  hope  they  would,  but 
they  may  not. 

Flow,  teammate 
positions 

TF,  K4 

What's  going  to  happen  in  the  groups  is  you're  going  to  get 
about  20  miles  apart  and  so  guess  what,  if  1  haven't  really 
illustrated  to  my  number  2  or  number  4  who  may  have  never 
executed  this  game  plan  before,  guess  what,  we're  cold  and 
we  turn  back  hot-there's  going  to  be  a  group,  20  miles  on  your 
nose,  Hot.  Guess  what,  that's  1  and  2... 

Communication¬ 

listening 

K4,  LM 

What  doesn't  happen  a  lot  is  all  that  instruction,  no  one  hears 
what's  going  on  because  they've  got  their  own  things  they're 
dealing  with  and  it  all  breaks  down  and  someone  ends  up 
targeting  someone  else. 

Cold  ops 

TF 

Cold  Ops  is  "ok,  we  lost  four"  or  "we're  all  cold  and  we  know 
they're  right  behind  us,  first  thing  we  have  to  do  is  figure  out 
where  everybody  is... everybody  shut  up,  viperl's  here,  1  and 

2  are  holding  hands. ..3  does  the  math  and  says,  Viper  3  is 
your  right,  2  o'clock,  10  miles.  Ok,  10  miles  going  this  way, 
everybody  head  0-7-0." 

Execution  Strategies 

Loss  of  radar 

K4,  TF 

#3  loosing  radar  is  a  lot  bigger  issue  because  they're  an 
element  lead  of  that  element,  they're  an  integral  part  of  the 
game  plan  because  #3,  at  least  the  way  we're  doing  things 
now,  is  really  a  fill-in  person.  I'll  put  2  &4  in  certain  spots  and 

3  catches  whatever's  coming  out  of  that. 

Missile  load 

K4,  DPC 

Just  like,  if  1  had  a  wingman,  and  we've  done  a  couple  pushes 
and  1  have  a  wingman  who's  out  of  missiles,  1  may  swap 
wingman  so  at  least  one  element  has  missiles.  Same  thing 
w/the  radar. 

Sorting,  prioritization 

K4,  MEE 

In  reality  there's  so  much  input,  you've  got  to  learn  to  prioritize 
that  [sorting  confusion] 

Radar,  authority 

DPC,  K4 

1's  really  running  the  flight,  but  when  3  &4  get  on  their  own,  3 
controls  4's  radar  so  there's  a  certain  level  of  autonomy  there 
that  you  have  to  respect,  but  if  you're  going  in  as  4,  1  now 
becomes  "hey,  you  do  as  1  tell  you,  1  own  everybody's  radar 
technically"  and  if  1  hear  3  doing  something  1  don't  want,  1 
hopefully  have  corrected  that  in  the  brief  so  that  everybody's 
radar  goes  where  it  should  and  3  directs  4  correctly  and  if  he 
doesn't,  then  1  correct  him  if  1  recognize  it. 

Building  the  picture 

K4 

Me  as  a  leader  going  into  a  tacts-formation  it's  my  job  to  build 
the  3D  picture  for  everyone  else  in  the  flight. 

Radar  position 

K4 

So  where  1  want  my  radar  and  my  wingman's  radar,  2  or  4, 
being  the  leaders  is  1  float  the  trailers  and  1  identify  them 

Spiked 

TF,  MS 

These  things  at  the  bottom  are  what  usually  get  left  out. 

Because  some  people  go  "help,  I'm  on  fire,  I'm  spiked"  but  the 
first  thing  is  survive,  then  element  mutual  support,  then  get 
untargeted  strikers  then  the  next  threat 

Surviving 

TF,  SA 

Surviving  may  be  running  towards  bad-guy  land  first  because 

1  know  there's  bad  guys  that  1  missed  back  this  way,  so  1  may 
have  to  run  this  way 

SA,  communication 

MS 

And  then  you  go,  ok,  now  what,  where  am  1,  "viper  4  is  blind, 
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bullseye  1-8-0  15"  everybody  goes,  ok,  he's  blind,  we  need  to 
get  back  the  element  mutual  support  here. _ 


Comm  priority 

MS 

No  one  else  has  anything  like  this,  no  one  is  defensive,  no 
one  is  threatened,  that  already  happened,  no  one  needs  a 
spike  range,  no  one's  declaring,  so  ok,  you  guys  have  the 
comm. 

Knowledge  that  Supports  Execution 

GCI  capabilities 

DTC,  MS 

He's  (#3)  not  going  to  be  able  to  know  what  that  range  is  or 
how  close  they  are  since  he  doesn't  have  a  radar.  In  this 
situation  it's  very  important  to  tie  GCI  in.  This  is  why  in  the 
mission  planning  part  of  it,  you  want  to  talk  about  these  things 
so  you're  not  just  rolling  in  the  air.  Now  GCI  can  play  a  role, 
tell  #3  how  close  they  are  to  the  nearest  threat,  really  help 
with  targeting  #4  in  that  situation. 

Mission  type,  radar 
status 

DPC, K4 

For  instance,  we  were  talking  about  the  differences  b/t  lane 
and  point  and  sweep,  this  [no  radar]  is  obviously  a  huge  deal 
if  we're  sweeping.  And  we've  got  to  go  forward  and  we're  not 
only  thinking  about  air  threats  and  ground  threats. 

Mission  type 

DPC,  MEE 

Where  this  also  makes  a  player,  and  1  know  we're  focusing  on 
DCA  lane,  now  if  2  and  3  loose  a  radar,  that  has  so  many 
more  ramifications  to  me  if  they're  dropping  bombs  because 
the  radar's  integral  to  that  as  well. 

Mission 

type/commander's 

intent 

DPC 

If  1  loose  two  of  them,  now  I’m  a  2-ship  and  1  really  loose  my 
lethality  and  1  need  to  think  about  do  1  really  need  to  be  there, 
period. 

Commander's  intent 

DPC 

In  a  DCA  lane,  I'm  probably  going  to  be  medium  [level  of  risk] 
which  means  I'm  probably  not  going  to  go  to  any  merges  that 

1  don't  have  to  or  that  1  don't  think  1  can  survive  out  of 

Commander's  intent 

DPC 

Well  no,  if  I'm  in  a  point  defense  and  I've  got  mom  or  dad  and 
the  kids  back  on  the  ground,  then  I'm  going  to  go  to  any 
merge  that  1  have  to  and  then  my  risk  level  is  high. 

Contingency  planning 

PE,  LM 

And  you  have  to  see  situations  and  be  presented  with  things 
or  you  need  to  have  thought  them  out  so  well  beforehand  that 
reacting  to  them  is  second  nature. 

Teammate 

characteristics 

DTC 

Well  guess  what,  it'll  get  you  in  trouble  a  lot  too  because  there 
may  be  this  golden  piece  of  information  that's  out  there  that 
people  are  afraid  to  say  because  "last  time  1  said  something 
about  that,  1  was  wrong,  and  1  got  crapped  on  in  the  debrief". 

Limitations 

DTC 

Because  most  of  the  time  AWACS  does  not  hear  the  VHF 

Teammate  capabilities 

DTC 

1  need  to  know  who,  what  capabilities  are  out  there,  are  you 
guys  HARM  shooters,  or  are  you. ..what's  your  weapons  load- 
out,  what's  our  weapons  load -out,  all  that  stuff. 

Team  capabilities 

K4 

F-16s  are  limited  a  little  bit  because  of  their  ID  capability  and 
not  necessarily  able  to  ID  friendly 

Team  capabilities 

K4,  DTC.  LM 

It  happens  here  a  lot  and  guys  a  lot  of  times  chalk  it  up  to 
visibility,  or  else  they  would've  been  able  to  see.  Well,  guess 
what,  you  wouldn't  have  been  able  to  see  and  you  wouldn't 
have  been  able  to  tell  that  was  an  F-16  versus  an  F-18  versus 
a  whatever  because  you're  not. 

Capabilities 

DTC,  K4 

So  much  of  this  goes  back  to  knowing  who's  out  there. 

Team  capabilities 

DTC,  TF 

So,  it  kind  of  goes  back. ..mostly  it  happens  when  you're  in 
the  big  packages  and  you  don't  know  where  other  groups  or 
entities  are. 

Procedure 

K4,  SA 

And  a  lot  of  that  stuff,  you  have  to  know,  getting  the  element 
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back  together,  will  happen  on  VHF.  All  the  other  stuff  will 
happen  on  UHF,  you  have  to  listen  to  both,  know  where  to 
talk  on  which  one,  what  you'd  say  and  then  Viper  3  would  say 
"ok,  I’m  10  miles,  you're  right  4  o'clock,  come  back  right..." 
and  now  hopefully  you'll  get  back  together. 

Enemy  threat 
capabilities 

ET,  K4,  LM 

But  if  we  bring  out... we  do  this  in  our  Intel  scenario,  "you're 
going  to  have  maneuvering  bandits,  they're  AMRAM  aware" 
or  whatever  the  case  may  be,  so  they're  going  to  try  to  defeat 
you. ..so  now  you  need  to  think  about,  we  could  bring  out  "hey 
guys,  think  about  your  missile  WRM,  how  many  missiles  you 
have  in  the  flight." 
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SME  2 


Description  of 
Interview  Content 

Knowledge 

Categories 

Interview  Content 

What-lf  Examples 

Alibis 

■  Practice  as  #2 
becoming  #4 
because  of: 

o  experience  mix 
o  resource 
distribution  in 
the  two,  two 
ships 

Listening  and  comms 
are  likely  to  be  a 
training  issue  because 
call  signs  will  be 
different 


Present  a  situation  in 
which  flight  lead  loses 
radar.  Present  from 
these  perspectives: 

■  flight  lead  has  to 
use  wingman’s 
radar  for  tactical 
decision  making. 

■  wingman  has  to 
communicate  his 
radar  information  to 
the  flight  lead  to 
support  lead’s 
tactical  decision 
making 


Because  of  poor 
radar  mech,  or  poor 
comms,  both  1  and  2 
are  sorted  to  the 
same  (leader  or 
trainer).  Ask:  Why  is 
this  bad?  What  can 
be  done  to  be  aware 
of  this  situation? 


Based  on  a  situation 
presented,  ask, 
“Based  on  sort  matrix, 
where  is  your  sort 
going  to  be?” 


Every  once  in  a  while  a  guy  won't  have  a  radar.  If  you 
have  two  bad  radars,  now  you  may  have  to  start  swapping 
within  the  elements.  Fifty  percent  of  your  4-ship  probably 
won’t  happen,  but  I  can  see  one  (alibi)  happening  and  then  it 
really  depends  on  who  you’re  flying  with--the  experience 
level. 

When  switching  wingmen.  There  may  be  some  comm 
issues  as  far  as  using  the  wrong  call  sign. ..the  guy  needs  to 
make  a  mental  switch  from  okay  now  I  am  number  4  versus  2 
or  what  ever. 

If  we  lose  radar  or  someone  has  no  missiles  or  like  one 
element  has  no  missiles  or  no  radar,  we  will  get  someone 
with  a  radar  and  missiles  in  that  element.  And  that  is  pretty 
much  all  that  needs  to  be  said. 


Flight  lead  using  radar  of  wingman.  I'm  using  your  radar. 
I'll  make  the  tactical  decision  but  I  am  using  information  that  I 
listen  to  often,  what  you’re  saying  to  make  my  decisions  on. 


So,  one  of  you  can  be  so  clueless  that  he  hasn’t  been 
listening  or  doesn’t  have  the  radar  mech  to  know  who  the 
leader  and  trailer  is,  Or,  two,  he  just  says  the  wrong  thing 
because  of  a  brain  fart.  Now  he  will  be  sitting  there  not  really 
knowing.  What  that  may  mean  is  that  if  you  think  that  he  has 
sorted  on  the  leader,  okay,  which  is  two  airplanes,  one  or  two 
miles  apart.  He  says  he  is  sorted  on  the  leader  but  he  really 
sorted  to  the  trailer.  Okay,  he  may  bring  the  radar  into  the 
trailer,  what  about  if  the  leader  goes  like  this  (splits).  Now  he 
goes  away.  Because  I  depend  on  him  to  tell  me  what  is 
going  on  with  certain  individuals.  Maybe  they  went  like  this 
and  started  splitting  and  I  didn't  know  that  because  all  of  our 
radar  went  with  this  one  guy.  Obviously  that  is  a  problem 
because  we  have  no  SA  on  that  free  fighter.  That’s  not  good. 
So,  it  does  come  down  to  communication. 


Sometimes  you  draw  little  pictures  up.  I’ll  draw  pictures  up 
and  go,  “Okay,  if  we  see  a  two-ship  line  abreast,  what’s  the 
sort  going  to  be?”  And  you  just  talked  about  your  sort 
matrix — side/side,  near/far.  So,  where’s  sort  the  going  to  be 
to?  And  you  look  right  at  him.  And  he  goes,  “Okay  my  sorts 
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Show  an  engagement 
and  the  beginning  of 
the  cold  ops  period. 

■  Ask,  what  would  be 
your  priority  be 
during  cold  ops? 
What  information 
would  you  get? 
What  would  your 
game  plan  be? 

■  Show  Darkstar 
making 
inappropriate 
comms  (provides 
information  that 
flight  lead  should 
know.)  Ask,  why  is 
this  a  problem? 


Ask:  Would  you  go  to 
the  merge  2v4/1  v2? 


Ask:  You  have  a  tally 
on  a  bogey  group, but 
your  wingman 
doesn’t.  You  know  he 
doesn’t.  Do  you 
continue  to  the 
merge? 

■  What  if  group  is 
aware? 

■  What  if  group  is 
unaware? 


DP&C  ,  SA,  L, 
ET 


Show  radar 
presentations  for 
different 


going  to  be  there.”  And  you  go  “Very  good." 


Guys  that  do  cold  ops  well  build  a  mental  geometry  about 
what’s  behind  them  quicker  and  they  come  up  with  a  game 
plan.  You  got  to  go,  “where  are  we  in  relation  to  each  other?” 
So  the  typical  game  plan  is  once  we  know  we  are  all  cold,  say 
“viper  3  posit.”  “Viper  3,  is  at  your  right  4  o’clock,  7  miles.” 

Once  you  know  that  you  may  sit  there  and  go,  “viper  1  is  2  by 
2  by  gun.”  or  whatever  your  gas  is.  So  everybody  is  going  to 
do  that.  Why  is  that  important,  because  if  I  have  a  high  fast 
flyer,  or  if  I  have  a  hostile  group  and  I  got  zero  missiles  and 
zero  missiles  or  zero  120s,  or  2  120s  and  1  120,  that  means 
I’m  not  going  to  turn  around  and  try  and  shoot  those  guys 
because  I  don't  have  missiles. 

Then  you  want  to  come  up  with  a  picture.  Say,  “Darkstar, 
Viper  1 ,  picture.” 

“Viper  1,  picture,  two  groups,  azimuth.  North  group 
bullseye  x-y-z  hostile,  South  group  x-y-z  hostile” 

“Darkstar,  Viper  1 ,  BRAA  to  north  group.”  (BRAA  is 
bearing,  range,  altitude,  aspect) 

“BRAA  from  viper  3  to  south  group.” 

Then,  basically  you  come  up  with  a  gameplan. 

“Viper’s  game  plan  is  viper  3  you’re  going  to  delouse 
north  group  for  viper  1 As  soon  as  you  shoot  them,  turn 
back  cold  and  run  away,  and  we’ll  come  up  with  the  next 
game  plan.” _ 


Is  it  addressed  all  the  time  with  the  VID  thing,  that’s  a 
tough  question.  It  is  probably  not  addressed  as  much  as  it 
needs  to  be  addressed.  If  I  have  tally  on  those  guys,  and  my 
wingman  doesn't,  as  long  as  I  know  he  does  not  have  tally,  I 
can  continue  on  and  have  fairly  decent  lethality  at  the  merge. 
If  I  go  “viper  1  tally,  two  ship,  my  nose,  3  nm.”  Viper  2  does 
not  say  anything,  which  means  he  doesn’t  see  them.  Ill  sit 
there  and  merge  the  guy  and  say  “Viper  2,  merged  hostile.” 

At  that  point,  if  he  sees  me,  he  should  see  another  guy. 

Which  means  I  expect  the  guy  I’m  merged  with  to  blow  up(?). 

I  may  not  be  able  to  get  a  missile  off,  but  my  wingman,  at  that 
point,  should  be  able  to  go,  “Ah  ha,  I  seen  him  now”  and  then 
as  soon  as  we  pass,  he  shoots.  And  then,  we're  going  to  flow 
to  the  other  guy  and  kill  him.  So,  one  guy  should  die  fairly 
quickly. 

Now,  if  it  doesn't  happen  like  that,  then  I'm  going  to  be  in  a 
situation  where  I’m  BFMing,  dog  fighting,  two  guys,  and  I’m 
just  hoping  my  wingman  doesn't  get  shot.  I’m  trying  to  fly  his 
airplane  for  him.  That's  an  extremely  difficult  situation. 


Your  acceptable  level  of  risk  is  whether  I'm  going  to  go  to 
the  merge  or  not.  But  what  is  an  acceptable  merge?  If 
they’re  totally  stupid  (the  bogeys),  if  there  are  two  of  me,  I’ll 
take  twice  as  many  of  them.  If  they're  aware,  though,  I’m 
only  taking  same  numbers. 

Most  guys  will  brief,  “if  they’re  unaware,  we’ll  go  one  of  us, 
two  of  them,  and  one-to-one  if  they’re  aware.”  Say,  they’re 
aware  if  there's  an  altitude  trend  towards  you,  if  they  spike  or 
they’re  showing  hot  aspect.  Most  guys  will  define  awareness. 


Draw  a  few  pictures,  because  drawing  a  few  pictures  doesn't 
take  a  lot  of  time,  and  it  just  reinforces  the  idea.  Sometimes 
I  will  draw  it  for  a  brand  new  guy,  which  I  won't  really  do  in 
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configurations.  Ask  an  operational  unit.  Sit  there  and  go,  “we  have  either  two 

we  have  either  two  ship  line  abreast  or  a  two  ship  trail.  What  is  that  going  to 

ship  line  abreast  or  a  look  like  on  your  radar?"  You  may  see  something  like  (draw) 

two  ship  trail.  What  is  that  versus  something  like  that  (draw).  You  would  do  that 

that  going  to  look  like  for  a  brand  new  guy?  Yeah,  not  that  we  probably  don’t 

on  your  radar?"  need  to  do  more  of  it,  as  a  whole.  Sometimes  that  is  a 

_ _  difficult  thing  to  do. _ 


Plan  Elements 

Discuss  swapping 
wingmen 

DP&C,  MS 

It  is  not  like  you  are  going  to  sit  there  and  go,  “if  3  loses  radar, 
I’m  going  to  be  talking  to  you  a  lot.”  1  mean,  that’s  going  to  be 
obvious.  When  you  swap  wingman,  though,  people  will  talk 
about  that. 

If  we  lose  radar  or  someone  has  no  missiles  or  like  one 
element  has  no  missiles  or  no  radar,  we  will  get  someone  with 
a  radar  and  missiles  in  that  element.  And  that  is  pretty  much 
all  that  needs  to  be  said. 

Sort  plan/target  plan 

SA 

Typically,  well  have  a  target  plan.  Most  people  will 
always  talk  target/sort,  and  put  it  up  some  way  visually 
so  people  will  know.  So,  it  may  be  like  1  or  2:  side/side, 
near/next  near,  or  out  of  direction.  My  sort  plan  may  be 
side/side,  near/far,  then  if  we  are  going  to  a  VID  merge, 
maybe  like  trailer.  That  is  pretty  much  is  written  on  the 
board. 

Sort  plan/target  plan 

MS 

Sometimes  the  brief  is  better.  The  simpler  the  plan,  the 
easier  it  is  to  execute  by  everybody.  Remember  the  targeting 
and  sort  matrix  1  drew  up  there?  It  should  be  easily  defined. 

So,  if  guys  are  sticking  to  that,  everyone  knows  if  groups  are 
in  azimuth,  my  radar  goes  here  and  your  radar  goes  there. 

Contact  range 

DP&C 

The  hard  part  is  looking  on  your  radar,  getting  that  SA,  or 
listening  to  comms  that  maybe  you  can't  detect  that  group. 

That  is  where  the  hard  part  is.  In  the  mission  planning  part, 
you're  going  to  know  what  your  contact  range  is.  That's  going 
to  drive  some  of  your  tactics.  That’s  going  to  drive  shot 
range,  that’s  going  to  drive  how  far  in  you  might  go. 

Level  of  risk 

DP&C 

Is  that  (i.e.,  level  of  risk)  something  you  would  talk  about 
in  preparation  or  the  brief?  Yep,  you  will  know  that.  In 
combat,  that  is  told  to  you.  You  will  know.  The  General  goes, 
“You  will  die  for  your  country  today  no  matter  what”  or  “guys, 
do  not  die  for  country  today,  do  not  go  to  merges.”  And  we 
know  that,  and  that  is  part  of  their  mission  plan.  That  is 
spelled  out  in  special  instructions. 

Resources  spelled  out 
in  the  ATO 

L,  ET 

And  also  what  if  1  knew  there  were  surface  to  air  missiles 
back  here  that  are  good.  Well  maybe  1  don't  have  to  accept 
this  anymore,  maybe  1  could  go,  “I’ve  done  everything  1  can, 
you  know.  I’m  going  to  exit  the  fight  and  you  gotta  take  out 
the  bad  guys  coming  through  this  area.”  So,  that’s  all  part  of 
the  game,  not  the  game,  the  plan.  So,  it’s  all  in  the  air  tasking 
order,  I’m  talking  combat  now,  things  that  we  are  doing  right 
now  in  various  places  of  the  world.  You’ll  know  all  the  assets 
that  you  are  dealing  with 

Define  risk  area 

T&F 

Guys  that  define  the  risk  area  have  a  very  good  tactics  flow. 
You  may  go  from:  skate  to  short  skate  to  bonsai,  based  on 
where  the  bad  guys  are  showing  up. 

VID  situations 

DP&C,  ET,  L 

1  think  you  owe  it  to  yourself  to  brief  it.  Most  people 
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do.  Realistically,  though,  you’re  going  to  know  a  lot 
more.  You’re  going  to  know  whether  that  guy’s  hostile 
or  not. 

Should  VID  be  part  of  a  brief?  Let’s  say  you’re  going  to 
be  up  there  autonomously.  Say,  AWACS  is  going  to  be 
midnight  from  here  to  here,  or  off  station.  1  have  flown 
missions  like  that.  Then  you’re  going  to  have  to  do  that  (VID). 
But  you  are  always  going  to  talk  about  what  if  you  have  a 
bogey  group  versus  a  hostile  group.  You  don’t  ever  brief  just 
a  bogey  group  or  you  don't  ever  brief  just  a  hostile  group. 

Ordnance/threats 

ET 

Knowing  the  ordnance  you  have  and  who  you  are  fighting  is 
all  wrapped  into  the  mission  planning  part. 

Define  “awareness” 

DP&C,  ET 

Most  guys  will  brief,  “if  they’re  unaware,  we'll  go  one  of  us, 
two  of  them,  and  one-to-one  if  they're  aware.”  Say,  they’re 
aware  if  there's  an  altitude  trend  towards  you,  if  they  spike  or 
they're  showing  hot  aspect.  Most  guys  will  define  awareness. 

Leadership 

Directing 

MS 

Fifty  percent  of  your  4-ship  probably  won’t  happen,  but  1  can 
see  one  (alibi)  happening  and  then  it  really  depends  on  who 
you’re  flying  with--the  experience  level.  You  may  get  a  very 
experienced  wingman  on  1’s  wing,  then  you  go  “2  and  4  you’ll 
have  to  swap,”  and  now  that  guy  can  control  the  element. 

Planning  Strategy 

Simulation/Role- 

Playing 

SA 

Sometimes  you  draw  little  pictures  up.  If  this  is  a  good  way  to 
do  this  let  me  know,  if  it’s  not  then  1  can  change  it.  I’ll  draw 
pictures  up  and  go,  “Okay,  if  we  see  a  two-ship  line  abreast, 
what's  the  sort  going  to  be?”  And  you  just  talked  about  your 
sort  matrix — side/side,  near/far.  So,  where's  sort  the  going  to 
be  to?  And  you  look  right  at  him.  And  he  goes,  “Okay  my 
sorts  going  to  be  there.”  And  you  go  “Very  good." 

Visualization 

DP&C 

Being  at  Luke,  ...I’ll  draw  a  picture  up  of  the  radar  and  I’ll  go, 
“when  1  tell  you  when  to  sort,  anticipate  it  is  going  to  be  no 
later  than  15  nautical  miles  and  this  is  about  where  it  is  going 
to  be.  Remember  we’re  going  to  be  checked  off  to  one  side” 
so  1  draw  a  little  picture  on  how  to  do  that.  As  you  get  more 
experienced,  you  shouldn’t  really  have  to  say  that.  1  mean  as 
long  as  you  have  the  standard  sorting  plan,  “our  sort  today: 
side  to  side,  near  far,  or  unless  it  is  a  lead-trail  then  1  will  get 
the  trailer  and  you  get  the  leader.”  And  that  is  pretty  much 
standard  throughout  most  communities. 

SA,  MS 

Typically,  we’ll  have  a  target  plan.  Most  people  will  always 
talk  target/sort,  and  put  it  up  some  way  visually  so  people  will 
know.  So,  it  may  be  like  1  or  2:  side/side,  near/next  near,  or 
out  of  direction.  My  sort  plan  may  be  side/side,  near/far,  then 
if  we  are  going  to  a  VID  merge,  maybe  like  trailer.  That  is 
pretty  much  is  written  on  the  board. 

SA,  MS 

Just  go  over  the  comm.  It  is  always  important  to  really  pack 
the  comm.  Draw  a  few  pictures,  because  drawing  a  few 
pictures  doesn't  take  a  lot  of  time,  and  it  just  reinforces  the 
idea.  Sometimes  1  will  draw  it  for  a  brand  new  guy,  which  1 
won’t  really  do  in  an  operational  unit.  Sit  there  and  go,  “we 
have  either  two  ship  line  abreast  or  a  two  ship  trail.  What  is 
that  going  to  look  like  on  your  radar?"  You  may  see  something 
like  (draw)  that  versus  something  like  that  (draw).  You  would 
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Team  configurations  L,  MS 


Define  Risk  Area 


Know  team  member  L,  MS 

capabilities  and 

limitations 


Address  Cold  Ops 
procedures. 


DP&C,  SA 


do  that  for  a  brand  new  guy?  Yeah,  not  that  we  probably 
don’t  need  to  do  more  of  it,  as  a  whole.  Sometimes  that  is  a 
difficult  thing  to  do. _ 


Don't  count  numbers  and,  whenever  you  brief,  it  should  be 
generic.  It  is  2  v  x,  or  4  v  x,  or  8  v  x.  Don't  sit  there  and  go, 
“Today,  we're  doing  2  v  2”  so  you  only  draw  two  players  up 
there  the  entire  time.  ...When  you  talk  about  your  bogey 
game  plan,  and  you  have  a  hostile-bogey  group,  well  you 
don't  want  to  say,  I  shot  one  hostile  and  I  know  he's  dead. 
Now  I’m  going  to  go  to  the  bogey  group.  There's  only  one 
guy  there.  You  want  to  keep  practicing  what  you’re  going  to 
be  doing.  So,  I  think  most  guys  are  pretty  good  about  keeping 
it  2  v  x.  I’m  not  going  to  draw  on  the  board  this  one  guy. 

When  I  talk  about  a  bogey  game  plan,  I  want  to  go  up  to 
about  four  aircraft.  I  don't  want  to  go  to  the  merge  if  it’s  more 
than  that.  It’s  unacceptable. 


Teams.  In  combat  you’re  going  to  stick  inexperience  with 
experience,  and  so  forth.  So,  if  you  have  an  inexperienced 
flight  lead,  you'll  have  an  experienced  wingman. 


But  combat,  they’ll  take  a  look  at  who’s  flying  with  who  and 
then  they  will  appropriately  put  the  right  people  together 


Flying  with  same  team  member;  shared  SA.  Even  in  Red 
Flag,  you’ll  have  a  crew  you  that  you  fly  with  pretty  much  the 
whole  week.  When  you’re  with  people  over  a  series  of  days 
and  missions,  you  all  start  thinking  “I  know  what  Odie’s 
thinking”...  you  can  already  tell  what’s  going  to  happen.  That 
makes  it  pretty  nice. _ 


I’ll  draw  up  a. ..Being  at  Luke,  ...I’ll  draw  a  picture  up  of  the 
radar  and  I’ll  go,  “when  I  tell  you  when  to  sort,  anticipate  it  is 
going  to  be  no  later  than  15  nautical  miles  and  this  is  about 
where  it  is  going  to  be.  Remember  we're  going  to  be  checked 
off  to  one  side”  so  I  draw  a  little  picture  on  how  to  do  that.  As 
you  get  more  experienced,  you  shouldn’t  really  have  to  say 
that.  I  mean  as  long  as  you  have  the  standard  sorting  plan, 
“our  sort  today:  side  to  side,  near  far,  or  unless  it  is  a  lead-trail 
then  I  will  get  the  trailer  and  you  get  the  leader.”  And  that  is 
pretty  much  standard  throughout  most  communities. 


Guys  that  define  the  risk  area  have  a  very  good  tactics  flow. 
You  may  go  from:  skate  to  short  skate  to  bonsai,  based  on 
where  the  bad  guys  are  showing  up. _ 


Know  his  (AWACS')  limitations,  and  they  go,  “what  are  your 
limitations  today?”  Well,  I  have  a  hard  time  seeing  below  this 
area.  So,  you  need  to  know  that  because  you  may  dedicate 
somebody;  “hey  when  we  look  back  around,  you  look  low  with 
the  fighter  radar  versus  me  looking  at...  So  it  does,  if  you  talk 
to  them  I  think  they  need  to  know  their  limitations,  maybe 
comm,  limitations.  If  you  can’t  get  a  declaration  to  shoot 
somebody  because  you  are  flying  at  8  thousand  feet  but  ...15 
thousand  feet  I  could  have  hurt  them.  Maybe  knowing  that  is 
helpful. _ 


This  is  what  you  say — we’re  going  to  get  our  posit  down, 
then  you  give  a  comm  example,  then  you  do  a  skate  check, 
and  give  an  example,  then  we’re  going  to  do  a  picture,  and 
give  an  example,  then  I'm  going  to  maybe  ask  for  BRAA  so  I 
can  figure  out  where  are  they  exactly  behind  me,  how  far, 
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altitude.  And  that  will  help  us  formulate  a  game  plan. 

A  poor  brief  won’t  address  cold  ops  or  they  build  little  errors 
into  their  plan  because  they’ve  never  been  faced  with  that 
situation.  Or  some  guys  will  say,  I  don’t  want  a  picture,  I 

_ want  everything  in  BRAA. _ 

Execution  Challenges 

Communication  SA  (When  switching  wingmen.)  There  may  be  some  comm 

issues  as  far  as  using  the  wrong  call  sign. ..the  guy  needs  to 
make  a  mental  switch  from  okay  now  I  am  number  4  versus  2 

_ or  what  ever. _ 

SA  It  all  comes  down  to  comm.  And  it's  easy  to  say  comm. 

Communications  today  is  3-1  standard,  which  everyone 
should  know.  There  is  a  lead  group,  trail  group,  north  group, 
south  group.  There  is  a  leader,  there  is  a  trailer  in  the  group. 
So,  one  of  you  can  be  so  clueless  that  he  hasn't  been 
listening  or  doesn't  have  the  radar  mech  to  know  who  the 
leader  and  trailer  is,  Or,  two,  he  just  says  the  wrong  thing 
because  of  a  brain  fart.  Now  he  will  be  sitting  there  not  really 
knowing.  What  that  may  mean  is  that  if  you  think  that  he  has 
sorted  on  the  leader,  okay,  which  is  two  airplanes,  one  or  two 
miles  apart.  He  says  he  is  sorted  on  the  leader  but  he  really 
sorted  to  the  trailer.  Okay,  he  may  bring  the  radar  into  the 
trailer,  what  about  if  the  leader  goes  like  this  (splits).  Now  he 
goes  away.  Because  I  depend  on  him  to  tell  me  what  is 
going  on  with  certain  individuals.  Maybe  they  went  like  this 
and  started  splitting  and  I  didn’t  know  that  because  all  of  our 
radar  went  with  this  one  guy.  Obviously  that  is  a  problem 
because  we  have  no  SA  on  that  free  fighter.  That’s  not  good. 

_ So,  it  does  come  down  to  communication _ 

SA  So,  I  would  expect  him  to  call  “lock,  viper  2  lock,”  and  we’ll 

probably  read  a  bullseye  position.  That  way  I  can  move  my 
cursors  and  go,  “viper  2  you  are  sorted,  ...”  So,  through 
communications  that’s  the  only  way  I  know  who  you  really 

_ have,  unless  you  have  SADL,  SADL’s  great! _ 

SA  And  that  is  exactly  right.  A  lot  of  time  I  will  sit  there,  if  I  hear 

“viper  2  sorted”  then  that  means  it’s  in  accordance  with  the 
plan.  He  doesn't  know  who  he  has  I  want  to  hear  “viper  2 
locked  “  because  that  means  he  doesn’t  who  he  has  or  he’s 
unsure.  It  wouldn't  hurt  my  feelings  either  if  he  said  “viper  2 
sorted  southern”  because  I  go,  he  is  where  he  is  supposed  to 
be.  Because  if  I  heard  him  say  “viper  2  sorted  northern”  I’d  go 

_ he’s  on  the  wrong  guy. _ 

SA  It  really  happens  a  lot  in  cold  ops,  too.  You've  seen  that 

where  the  vipers  are  running  away  and  there's  guys  chasing? 

_ That  is  where  the  comm  nightmare  starts. _ 

Execution  Strategies 

Communication  Skills  SA  What  can  be  done  about  Targeting  the  same  group? 

(Obtaining  and  Listening  and  comm.  And  saying  the  right  thing  at  the  right 

Sending)  time.  The  comm,  ball  is  between  GCI,  viper  1 ,  and  viper  3. 

“Viper  2,  when  we  target  you  into  a  group,  I  want  to  hear 
“targeted,”  and  I  want  to  hear  what  fox  is  coming  off  your  rail 
and  who  you  have  a  fox  on  and  I  want  to  hear  crank 
direction.”  There  is  a  comm  ball  and  comm  cadance,  so  when 
_  I  hear  a  guy  go,  “Viper  2  targeted,”  and  I  know  by  looking  that 
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Quitting  a  plan 


DP&C,  SA 


Obtain  information  on 
leading  edge  of  threat 
group(s) 


Use  other  team 
member’s  radar 


Change  Element 
Composition 


Use  Level  of  Risk  to 
Support  Decision 
Making 


DP&C,  SA 


DP&C,  SA, 
MS 


MS,  L 


Use  Decision  Points 


Use  Cold  Ops 


DP&C,  SA 


he  should  be  shooting  based  on  range,  and  I  hear  “viper  2, 
fox  3  north  group,  northern,”  I  go,  alright,  (then)  everything’s 
quiet,  excellent.  See,  you’re  following  comm  cadance.  But  if 
I  go,  “Viper  2  you're  sorted,”  boom,  that  interjected  a  radio  call 
where  viper  3  may  be  trying  to  shoot. _ 


Don’t  have  comms  going  to  more  than  two  people.  Okay. 
“Viper  1  is  going  to  delouse  viper  3.  Viper  3  you  delouse  viper 
1 .  Viper  1  in,  viper  3  in.”  You  have  comm  trying  to  go  to  two 
different  guys  at  the  same  time. 


When  going  to  the  merge.  Sometimes  you  may  turn  away 
and  say  I  can’t  accept  this.  My  team  does  not  have  enough 
SA  to  go  to  the  merge.  One  of  the  most  important  things  to 
do  when  you  go  to  the  merge  is  to  get  the  tally  ho  on  the 
whole  thing.  It  does  not  make  me  feel  good  to  know  I  just 
have  tally  on  one  in  a  two  ship,  or,  even  worse,  a  four  ship.  If 
I’m  only  seeing  one  of  the  guys  I  want  to  attack,  then  jeez!.  I 
want  to  hear  from  my  wingman  a  tally,  and  who  he's  tally’d  to, 
so  we  can  go  in  with  as  much  SA  possible. _ 


Or  some  guys  will  say,  I  don’t  want  a  picture,  I  want 
everything  in  BRAA.  Some  guys  want  the  whole  picture.  I 
just  want  to  know  about  the  leading  edge.  I  want  to  kill  him 
(leading  edge),  run  away,  and  worry  about  the  other  guys 
later.  Some  guys  want  the  whole  picture.  Well,  they’ll  rapidly 
understand  why  that’s  not  good  when  they  turn  back  against  a 
five  group. 


I'm  using  your  radar.  I’ll  make  the  tactical  decision  but  I  am 
using  information  that  I  listen  to  often,  what  you’re  saying  to 
make  my  decisions  on. _ 


If  we  lose  radar  or  someone  has  no  missiles  or  like  one 
element  has  no  missiles  or  no  radar,  we  will  get  someone 
with  a  radar  and  missiles  in  that  element.  And  that  is  pretty 
much  all  that  needs  to  be  said. 


Your  whole  tactic  is  based  on  what  the  level  of  risk  is.  If  the 
level  of  risk  is  low  and  merges  are  not  acceptable  you  will  not 
go  to  the  merge.  I.e.,  if  it’s  a  bogey  group,  ideally  you  have  to 
go  down  or  VID  these  guys.  Are  you  going  to  accept  that 
merge  or  not.  Well,  low  risk,  do  not  go  to  a  merge,  it’s  like, 
no.  And  usually  that  decision  is  made  up  a  lot  higher.  But  if  I 
am  protecting  this  AWACS,  or  this  base,  or  whatever,  I  may 
sit  there  and  go,  “out  here,  I  may  not  accept  the  merge,”  but 
as  we  get  closer  I  go,  “I  gotta  do  it,  I  gotta  accept  this  merge 
now”. 


You  can  base  your  decision  on,  I’m  in  this  area,  I  have  to  go 
to  the  merge.  Some  people  may  not  make  a  definitive  thing 
like  this  and  go,  “well  if  you  pump  once  you  come  back  ...” 
and  now  we’re  being  squeegeed  in  the  corner.  You  know  you 
need  to  quit  and  fight  with  everything  he  has.  I  like  the 
definition  of  a  range  or  whatever  it  gives  you  something  that  is 
right  there  not  going,  “well  I’m  not  really  in  threat.” 


Guys  that  do  cold  ops  well  build  a  mental  geometry  about 
what’s  behind  them  quicker  and  they  come  up  with  a  game 
plan.  You  got  to  go,  “where  are  we  in  relation  to  each 
other?”  So  the  typical  game  plan  is  once  we  know  we  are  all 
cold,  say  “viper  3  posit.”  “Viper  3,  is  at  your  right  4  o’clock,  7 
miles.”  Once  you  know  that  you  may  sit  there  and  go,  “viper 
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1  is  2  by  2  by  gun.”  or  whatever  your  gas  is.  So  everybody  is 
going  to  do  that.  Why  is  that  important,  because  if  I  have  a 
high  fast  flyer,  or  if  I  have  a  hostile  group  and  I  got  zero 
missiles  and  zero  missiles  or  zero  120s,  or  2  120s  and  1  120, 
that  means  I’m  not  going  to  turn  around  and  try  and  shoot 
those  guys  because  I  don’t  have  missiles. 

Then  you  want  to  come  up  with  a  picture.  Say,  “Darkstar, 
Viper  1 ,  picture.” 

“Viper  1,  picture,  two  groups,  azimuth.  North 
group  bullseye  x-y-z  hostile,  South  group  x-y-z  hostile” 

“Darkstar,  Viper  1,  BRAA  to  north  group.”  (BRAA 
is  bearing,  range,  altitude,  aspect) 

“BRAA  from  viper  3  to  south  group.” 

Then,  basically  you  come  up  with  a  gameplan.  “Viper’s 
game  plan  is  viper  3  you’re  going  to  delouse  north  group 
for  viper  1 .”  As  soon  as  you  shoot  them,  turn  back  cold 
and  run  away,  and  we’ll  come  up  with  the  next  game 
plan.” _ 


Stick  to  plan 

SA,  MS 

The  simpler  the  plan,  the  easier  it  is  to  execute  by  everybody. 
Remember  the  targeting  and  sort  matrix  1  drew  up  there?  It 
should  be  easily  defined.  So,  if  guys  are  sticking  to  that, 
everyone  knows  if  groups  are  in  azimuth,  my  radar  goes  here 
and  your  radar  goes  there.  That’s  pretty  simple.  The  hard 
part  is  looking  on  your  radar,  getting  that  SA,  or  listening  to 
comms  that  maybe  you  can’t  detect  that  group.  That  is  where 
the  hard  part  is.  In  the  mission  planning  part,  you’re  going  to 
know  what  your  contact  range  is.  That’s  going  to  drive  some 
of  your  tactics.  That’s  going  to  drive  shot  range,  that’s  going 
to  drive  how  far  in  you  might  go. 

Knowledge  that  Supports  Execution  &  Planning 

Execution  Templates 

L 

Fifty  percent  of  your  4-ship  probably  won't  happen,  but  1  can 
see  one  (alibi)  happening  and  then  it  really  depends  on  who 
you're  flying  with--the  experience  level 

Timeline  Awareness 
and  Own  Capabilities 
and  Limitations 

L,  DP&C 

I’ve  heard  that  from  the  Army,  too,  about  not  being  overly 
committed  to  your  plan.  That’s  hard  to  do--back  away 
from  a  plan.  It  is  hard.  You  see  these  airplanes  and  they 
have  no  clue  we’re  there.  But  you  still  can’t  shoot  them  until 
you  see  what  they  are.  So  here  you  go.  You  have  a  great 
position,  but  you  don't  have  all  the  tally  hos,  but  you  go  in 
there.  And,  they  go,  “someone  is  going  to  attack  me,”  and 
they  turn,  and  the  one  you  don't  see  is  going  to  kill  you.  Even 
though  up  until  the  time  they  turned  you  were  in  an  incredibly 
offensive  position.  All  of  the  sudden  the  hunter  became  the 
hunted. 

Knowledge  of  Element 

MS,  L 

If  you  have  two  bad  radars,  now  you  may  have  to  start 
swapping  within  the  elements.  Fifty  percent  of  your  4-ship 
probably  won't  happen,  but  1  can  see  one  (alibi)  happening 
and  then  it  really  depends  on  who  you’re  flying  with--the 
experience  level.  You  may  get  a  very  experienced  wingman 
on  1  ’s  wing,  then  you  go  “2  and  4  you'll  have  to  swap,”  and 
now  that  guy  can  control  the  element. 

SA 

(common  ground)  Even  in  Red  Flag,  you’ll  have  a  crew  you 
that  you  fly  with  pretty  much  the  whole  week.  When  you're 
with  people  over  a  series  of  days  and  missions,  you  all  start 
thinking,  “1  know  what  Odie's  thinking”...  you  can  already  tell 
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_ what's  going  to  happen.  That  makes  it  pretty  nice. _ 

Knowledge  of  MS,  DP&C  Part  of  the  decision-making  would  be  to  retrograde  the 

Distributed  Teams  asset  you  are  protecting?  Yes.  There  may  be  a  range.  If 

Capabilities  it  is  AWACS,  he’ll  retrograde  himself.  He'll  slide  or  at  least, 

say,  “Dude,  I'm  out  of  here  man.”  Wooh,  he’s  gone.  “Thanks 
for  helping  me  out,  by  the  way.”  So,  you  can  go  ahead  and 
direct  people,  slide  this  way  or  move  this  way.  That  is  a 
higher  level  of  understanding  than  I  know  you  are  going  to 

_ find  most  in  Viper  communities. _ 

L,  DP&C,  MS  Know  his  limitations,  and  they  go,  “what  are  your  limitations 
today?”  Well,  I  have  a  hard  time  seeing  below  this  area.  So, 
you  need  to  know  that  because  you  may  dedicate  somebody; 
“hey  when  we  look  back  around,  you  look  low  with  the  fighter 
radar  versus  me  looking  at...  So  it  does,  if  you  talk  to  them  I 
think  they  need  to  know  their  limitations,  maybe  comm, 
limitations.  If  you  can't  get  a  declaration  to  shoot  somebody 
because  you  are  flying  at  8  thousand  feet  but  ...15  thousand 

_ feet  I  could  have  hurt  them.  Maybe  knowing  that  is  helpful. 

SA  This  is  what  you  say — we’re  going  to  get  our  posit  down, 

then  you  give  a  comm  example,  then  you  do  a  skate  check, 
and  give  an  example,  then  we’re  going  to  do  a  picture,  and 
give  an  example,  then  I’m  going  to  maybe  ask  for  BRAA  so  I 
can  figure  out  where  are  they  exactly  behind  me,  how  far, 
altitude.  And  that  will  help  us  formulate  a  game  plan.  Dark 
star,  AWACS  is  listening,  so  they  are  part  of  it.  Do  you  want 
them  to  be  quiet  during  this  unless  there’s  something 
you  definitely  should  know?  Yes,  for  example,  I  know  I  left 
at  12  miles.  If  they  are  not  puncturing  our  threat  range,  I 
don’t  need  to  hear  about  that.  Or  if  you  called  the  threat  and  I 
turned  cold,  I  don’t  need  to  hear  about  that  again.  I  know 
they’re  back  there.  So  don’t  be  saying  “Threat,  BRAA, 
bullseye...  “  I  know  they’re  back  there.  I’m  running  away 
from  them.  So  tell  me  something  I  don’t  know.  Keep  quiet. 

_ So  get  BRAA  to  that  group  when  you're  ready  to  listen. _ 

Blue  on  Blue  SA,  T&F  When  I  go  to  the  merge  and  my  wingman  has  no  clue.  That 

Situations  is  most  likely  where  a  frat  will  happen.  I  would  say  VID 

merges  are  the  worst  ones.  When  you  go  into  that  merge 
with  low  SA,  and  you  don’t  kill  someone  right  off  the  get  go, 
you  end  up  turning  around.  As  soon  as  that  thing  turns,  your 

_ whole  SA  starts  shrinking. _ 

When  to  go  to  the  DP&C,  ET  Your  acceptable  level  of  risk  is  whether  I’m  going  to  go  to  the 

merge  merge  or  not.  But  what  is  an  acceptable  merge?  If  they’re 

totally  stupid  (the  bogeys),  if  there  are  two  of  me,  I’ll  take 
twice  as  many  of  them.  If  they’re  aware,  though,  I’m  only 
_ taking  same  numbers. _ 
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APPENDIX  C 


Background  Interview  Data 
Obtained  from  One  Air  Combat  SME 

Notes.  Knowledge  category  codes  are  defined  in  Table  3.  Interview  content 
represented  in  bold  font  are  interviewer  verbalizations. 


Description  of 
Interview  Content 

Knowledge 

Categories 

Interview  Content 

What-lf  Examples 

Present  a  situation  in  a 
grinder  in  which  #2  did 
not  follow  the  game  plan 
and  forces  a  VI D 
situation. 

SA,  MS, 

DP&C 

That’s  what  I’m  talking  about,  where  he  didn't  follow  the 
game  plan  and  if  he  doesn’t  say  anything  and  all  of  a 
sudden,  me  as  number  one  come  wheeling  around,  and 
now  I’m  going,  “hey,  what  is  this?”  Now  1  may  not  be  able  to 
employ  the  BVR,  because  now  1  got  friendlies  and  hostiles 
all  mixed  up  in  the  same  group,  then  that’s  bad. 

So  how  would  in  a  situation  like  that,  if  you’ve  been  in 
one,  what  does  everybody  do  to  get,  you  know,  what  do 
they  do?  Ah  well,  if  they're  mixed  up  then  three  and  four 
better  run  like,  excuse  me,  run  like  hell  and  then  now  I'm 
going  to  have  to  really  use  some  com  to  decipher  who  is 
who,  and  sometimes  that  can  happen,  sometimes  it  can't 
happen  and  it  may  force  a  visual  identification  problem.  So, 
now  your  whole  focus  is  you’ve  got  to  have  your  guys 
survive,  although  you’d  rather  club  them  over  the  head  with 
a  club.  But,  now  you're  going  to  have  to  save  them  totally 
come  off  your  game  plan,  you  have  to  forget  about 
everything  else  that’s  going  on  and  you're  going  to  have  to 
go  and  get  these  guys  off  of  number  three  and  four. 

Present  tactical 
situations  and  ask 
trainees  how  to  position 
themselves  to  be  in  an 
advantageous  situation 

DP&C,  T&F, 

SA 

Well,  interpreting  the  radar  in  where  do  1  need  to  flow  my 
flight.  To  put  myself  in,  in  an  advantageous  situation.  We 
always  see  problems  with  groups  doing  that,  usually  they 
can  target  correctly,  their  shot  doctrine  is  usually  pretty 
decent.  1  think  putting  their  flight,  though,  in  a  more 
advantageous  position  is  sometimes  a  little  bit  of  a  limiting 
factor,  so  they  start  off  from  a  position  of  disadvantage, 
whereas  if  they  would  have  flowed  their  flight  a  little  better 
then  they  would  have  been  in  a  position  of  advantage. 

Plan  Elements 

Discuss  what  to  do  if 
groups  are  maneuvering 

ET,  DP&C 

Yeah,  you’ll  talk  about  that,  a  lot  of  times  you’ll  sit  there  and 
go,  okay  if  groups  are  maneuvering,  we  are  just  going  to 
turn  around,  run  away  and  let  it  settle  out.  Because  again, 
you  know,  if  they’re  doing  exploding  cantaloupe  they  can't 
kill  you  and  they  can't  kill  whatever  they  wanted  to  kill, 
because  they  are  doing  all  those  funky  maneuvering  or 
whatever,  so  eventually  they’ll  have  to  come  back  hot,  and 
settle  down... 

Blue-on-blue 

DP&C,  SA 

You  talk  about  getting  good  targeting,  sorting  and  you  talk 
about,  you  know,  for  shot  doctrine,  you  always  have  to  have 
an  ID  and  a  clear  field  of  fire,  or  clear  avenue  of  fire.  Then, 
that’s  always  brought  up  pretty  much,  because  the  guy 
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hitting  the  tickle  button  has  to  know,  no  kidding,  if  that’s  a 

_ bad  guy. _ 

Commit  criterion  DP&C  ...whenever  you  have  a  DCA  mission  anyway,  you're  going 

to  have  to  figure  out  what’s  your  criteria  for  your  commit. 

And  part  of  that’s  going  be  what  ordinance  you’re  carrying 
and  what  you’re  really  protecting.  Because  it  may  be  a  DCA 
lane  defense  mission  or  a  DCA  point  defense.  I  may  be 
given  a  block  that’s,  I  don’t  know,  say,  fifty  miles  wide  and  a 
hundred  miles  deep,  or  whatever... 

So,  first  you’ve  got  to  figure  out,  you  know,  the  kind  of  air 
you’re  playing  in  and  what  kind  of  ordinance  you're  to  be 
carrying,  because  that’ll  have  some  effect  on  what  you’re 
dong.  And  also,  the  political  situation.  You  take  a  look  at, 
well,  what’s  out  commit  criteria,  as  far  as  you  know?  Is  it 
only  when  they  come  across  the  border?  Or,  if  they  commit 
a  hostile  act  anywhere?  Is  that  clearance  for  us  to  engage? 
And  things  like  that.  So  that’s  kind  of  first,  first,  I  guess, 
decision  point,  is  okay,  what’s  really  going  on  in  the  big 

_ picture. _ 

Big  picture  DP&C  So,  first  you’ve  got  to  figure  out,  you  know,  the  kind  of  air 

you're  playing  in  and  what  kind  of  ordinance  you're  to  be 
carrying,  because  that’ll  have  some  effect  on  what  you’re 
dong.  And  also,  the  political  situation.  You  take  a  look  at, 
well,  what’s  out  commit  criteria,  as  far  as  you  know?  Is  it 
only  when  they  come  across  the  border?  Or,  if  they  commit 
a  hostile  act  anywhere?  Is  that  clearance  for  us  to  engage? 
And  things  like  that.  So  that’s  kind  of  first,  first,  I  guess, 
decision  point,  is  okay,  what’s  really  going  on  in  the  big 

_ picture. _ 

Contingencies  DP&C  One  thing  contingencies  is,  right  off  the  bat,  is  gas. 

You've  got  to  worry  about,  are  we  going  to  have  a  tanker? 
Maybe  the  people  next  door  on  another  lane,  they  fell  out 
and  then  you  have  to  start  worrying  about  gas  and  covering 
their  lane. 

Another  one  is,  what  if  AWACs  goes  midnight  and  you  don’t 
have  them  anymore. 

Another  one  is  weather,  you  know,  what  are  the,  the 
contrail  levels,  what  are  the  winds  and  what  are  the  clouds? 

What  if  somebody  has  a  problem  and  they  have  to  go 
back?  So  you  have  to  plan  for,  if  I  have  a  four  ship,  maybe 
being  a  three  ship. 

And  then  what  happens  if  like,  you  know,  number  one 
and  number  three  in  particular,  what  if  their  radar  goes  bad. 
So  one  of  the  airplanes  has  some  sort  of  an  aircraft  system 

_ limitation. _ 

Contingencies -AWACS  DP&C,  MS  ...let’s  talk  about  the  AWACS.  If,  let's  say  that  I  have  a  four 

ship,  well,  if  we're  all  going  the  same  way,  then  eventually  in 
my  CAP  I’m  going  to  have  to  turn  around  and  run  away,  and 
go  away  from  them  for  a  while.  If  we  don’t  have  AWACS, 
we  may  not  see  something  that  pops  up  on  the  scope,  okay, 
so  if  I  know  that  I  don’t  have  AWACS  I  may  have  to  adjust 
the  kind  of  formation  that  I’m  holding  my  CAP  in.  So  instead 
of  all  four  of  us  going  down  track  the  same  and  then  turning 
cold  and  running  away  then  turning  hot  and  running,  I  may 
put  two  guys  going  cold  while  the  other  two  guys  are  going 
_ hot.  Now  my  radars  are  out  there  looking,  so  I  always  have 
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somebody  looking  down  track. 

Well,  it  goes  back  to  my  tactic  of,  do  I  want  to  employ  us  as 
an  entire  four  ship  or  not.  If  there's,  I  may  not  be  able  to 
react  quite  as  fast  if  I’m  broken  down  into  two  ships  and  I 
want  to  come  out  as  a  four  ship.  It  shouldn’t  be  that  much  of 
a  limiting  factor  though,  if  two  ship  is  running  hot  they  see 
something,  they  can  always  call  it  out,  great,  so  everybody, 
even  though  my  radar  is  not  on  them  and  I’m  going  cold,  I 
know  about  them,  and  then  what  I  could  do  is  sit  there  and 
go,  okay,  well,  now,  I’m  going  to  turn  back  hot  and  hey 
number  three  I  want  you  to  rejoin  on  me  now,  so  we  can  go 
up  as  a  four  ship.  So,  it’s  not  that  big  of  a  deal  but  we  can 
get  at  least,  you  know,  radars  down,  some  radars  down 
there  and  then  we  can  come  up  with  a  game  plan.  The 
biggest  thing  about  no  AWACS  though,  at  least  for  F-16’s  is 
that  the  identification  of  the  bad  guys,  that’s  going  to  be  a 

_ limiting  factor,  and  I  won’t  go  more  into  that. _ 

Contingencies-radar  DP&C,  MS  If  the  radar  goes  bad  then  I  may  have  just  lost  a  lot  of  my 

missiles,  so  you  need  have  a  plan.  For  example  if,  number 
three  and  number  four  are  having  problems  with  their  radar, 

I  may  want  to  split  my  element  up  so  that  I  have  one  good 

_ radar  per  element. _ 

AWACS  role  MS  ...  the  most  successful  teams  are  the  ones  that  keep 

AWACS  in  the  loop  and  AWACS  keeps  them  in  the  loop  and 
there’s  not  too  much  blabbing  going  on  so,  if  everybody 
does  what  they  are  supposed  to  then,  then  it  goes  a  lot 
smoother.  AWACS  is  listening  for  any  groups  that  haven’t 
been  targeted  and  then  they’ll  remind  the  flight  lead,  “hey, 
guys  did  you  forget  about  this  guy.”  And  again,  he  is  part  of 
the  team  so  he’s  carrying  his  contracts  out  also. 

Is  any  of  that  ever  part  of  the  brief  or  part  of  the  plan, 
like,  just  reminding  like,  the  fighter  pilots  to  give  the 
AWACS  certain  information  at  a  certain  time?  Or  is  it, 
kind  of,  implicitly,  implicit  and  gained  with  experience? 

No.  It  should  be  part  of  the  brief.  But,  it  should  be  standard. 

I  mean,  AWACS  has  set  standards,  and  if  the  flight  lead 
wants  to  change  the  set  standards,  then  he  needs  to  brief 
them.  The  guys  that  do  the  best  are  the  ones  that  have  a 

_ very  good  working  relationship  with  the  AWACS. _ 

Use  of  standards  SA  There  are  always  standards.  So,  if  you  know  that  AWACS  is 

operating  off  of  the  latest  standards  and  you  are  operating 
off  of  the  latest  standards,  then  everything  should  be  good. 
But,  you  can  always  air  brief  a  guy,  saying  hey  I  want  this, 
you  know,  if  I  call  for  this,  I  want  this.  And  that’s  okay,  too. 
And  the  guy  on  the  console  needs  to  be  flexible  in  that  but, 

_ the  best  thing  is  to  stick  with  the  standards  and  use  them. 

Shot  doctrine  ET  So,  in  the  merge  phase,  as  far  as  the  pre-planning,  what’s 

extremely  important  is  your  shot  doctrine.  And  that's  going 
to  come  off  of  your  pre-planning. ...If  you’re  seeing  them 
doing  some  other  stuff,  you  can  sit  there  and  go,  wow,  I'm 
not  going  to  make  it.  They’re  running  me  out  of  gas,  or 
they’re  running  me  out  of  missiles,  and  you  can  sit  there 

_ now,  change  your  shot  doctrine. _ 

Communications _ SA _ I  go,  “Guys,  instead  of  like  blabbing  all  the  time  just  shut  up 
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and  listen,”  because  a  lot  of  times  there’s  a  lot  of  repeat 
information  out  there  which  lowers  everybody’s  situational 
awareness.  The  comm  time  gets  very  compressed  in  some 
situations  so,  if  someone  doesn't  listen  to  what  someone 
just  said  and  he  has  to  ask  for  the  same  information,  we  just 
lost.  Plus,  the  reply  of  whoever  he's  asking  information  for, 
then  we  just  lost  maybe  ten  seconds  of  com  time. 

And  what  happens  in  that  ten  seconds  if  someone  needs  to 
make  a  break  call,  you  know,  because  somebody  that  they 
didn't  see  all  of  a  sudden  popped  up  at  their  six  o'clock.  But 
someone's  blabbing  on  the  radio  about  something  that  was 
already  said.  So  a  lot  of  it  is  listening  and  really  digesting 
_ the  information. _ 

Plan  and  Brief  Organization 

Decision  sequence  DP&C  So,  the  first  one  is  what  is  my  mission?  Is  it  DCA  lane  or 

DCA  point?  What  kind  of  ordinance  do  I  have?  And  how 
long  do  I  have  to  be  there? 

At  that  point,  now  I’m  going  to  start  figuring  out,  What 
kind  of  force  do  I  need  to  protect  that?  Can  I  do  it  with  a  two 
ship?  Can  I  do  it  with  a  four  ship?  Am  I  going  to  have  to 
cycle  on  and  off  the  tanker?  Am  I  going  to  have  to  be  really 
careful  of  the  amount  of  missiles  that  I'm  employing  because 
I  don’t  have  enough?  Things  like  that. 

Now,  once  I  have  figured,  what's  the  big  picture,  then  I 
have  to  go,  okay,  where  do  I  want  to  put  my  cap  in  the  most 
advantageous  position  so  I  can  be  in  more  of  a  offensive 

_ position  and  cover  all  the  airspace  that  I  need  to  cover? 

Mission  phases  DP&C  Do  you  plan  around  those  phases?  Yes,  we  already 

talked  about  the  cap,  and  that’s  going  to  be  based  on  what 
am  I  protecting,  and  things  like  that,  and  also  what  kind  of 
ordinance  I'm  carrying.  Okay,  the  commit,  we  talked  a  little 
bit  about  already.  At  what  point  am  I  going  to  sit  there  and 
go,  I  need  to  leave,  excuse  me,  I  need  to  leave  this  cap  and 
go  and  engage  these  people. 

The  merge  prep  is  nothing  more  than  getting  my 
appropriate  radars  down  there  to  target  the  threats  coming 
my  way.  The  merge,  is  BVR  is  happening  at  range,  so  a 
merge  may  be  nothing  more  than,  now  we're  starting  to 
employ  ordinance  on  these  guys  and  then  we’re  going  to, 
clean  up  the  merge,  see  if  our  missiles  take  them  out.  And 
post-merge  is,  where  am  I  moving  my  forces  when  I  come 
off  this  engagement?  Where  am  I  flowing  to?  Am  I  going  to 
flow  down  to  the  south  to  take  care  of  somebody  else  or  to 
maybe  get  rejoined  with  my  flight?  That’s  what  we’re  talking 
_ about  in  post-merge.  _ 

Variations  in  Planning  and  Briefing 

Incorporation  of  AWACS  SA  It  helps  them,  I  mean,  it  keeps  their  SA  up,  because  again, 

into  plans  they’re  part  of  the  team,  the  most  successful  teams  are  the 

ones  that  keep  AWACS  in  the  loop  and  AWACS  keeps  them 
in  the  loop  and  there’s  not  too  much  blabbing  going  on  so,  if 
everybody  does  what  they  are  supposed  to  then,  then  it 
goes  a  lot  smoother.  AWACS  is  listening  for  any  groups 
that  haven't  been  targeted  and  then  they’ll  remind  the  flight 
_ lead,  like,  hey,  guys  did  you  forget  about  this  guy.  And 
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again,  he  is  part  of  the  team  so  he’s  carrying  his  contracts 
out  also. 

It  should  be  part  of  the  brief.  But,  it  should  be  standard... 
the  guys  that  do  the  best  are  the  ones  that  have  a  very  good 
working  relationship  with  the  AWACS. _ 


Leadership 

Directing 

SA 

If  1  tell,  tell  a  tactic  to  my  flight,  1  want  them  to  do  that  tactic 
and  don't  go  off  the  game  plan.  It’s  kind  of  like,  you're  the 
quarterback  and  you’re  going  to  tell  a  guy,  look,  1  want  you 
to  run  ten  yards  down  the  field  and  then  turn  to  the  outside, 
towards  the  sideline,  I’m  going  to  throw  you  the  ball.  So,  he 
runs  ten  yards  and  goes,  to  the  inside,  it’s  going  to  be  a 
messed  up  play.  Well,  the  same  thing  happens  if  there’s  a 
breakdown  in  the  game  plan.  Someone's  not  executing 
what  they’re  supposed  to  do  then,  all  of  a  sudden,  my 
situational  awareness  goes  down  and  that’s  when  we 
potentially  have  a  blue-on  -blue  problem.  So,  the  guys  have 
to  execute  the  game  plan,  and  other  than  that,  really,  the 
biggest  thing  about  blue-on-blue  is  when  you  enter  a  merge, 
a  visual  merge... 

Prioritization 

L 

1  always  try  to  give  my  wingman  the  easiest  thing,  and  1 
leave  all  the  hard  decisions  for  myself.  So  1  tell  them,  “if  1 
tell  you  to  target  a  group,  that  group  is  yours,  you  own  that 
group,  you  tell  if  the  group  is  doing  anything.”  The  wingmen 
that  do  that,  you  know,  we  are  a  better  fighting  team;  the 
wingmen  that  don't  do  that,  then  you’re,  your  SA  goes  down. 

Planning  Strategies  that  Support  Execution 

Standardization 

SA 

Well,  they  should  be  sticking  to  the  standards.  You 
sometimes  go  over  them.  If  1  knew  that  everybody  in  room 
was  totally  familiar  with  standards,  1  could  sit  there  and  go 
“comm  standard,”  and  that  is  the  only  thing  1  have  to  say. 

But  typically  you’ll  brief  a  little  bit  about  it,  you'll  go  over 
some  examples  and  make  sure  that  everybody  is  on  the 
same  sheet  of  music. 

Execution  Challenges 

Communication  SA  Let’s  say  that  I  have  a  two  ship  that's  going  to  attack  that 


four  ship.  Well,  the  more  I  know  about  that  group,  then  the 
better  I’m  going  to  be  able  to  attack  it.  Let’s  say  that  I  only 
send  in  a  two  ship  though,  and  all  of  a  sudden,  I  find  myself 
in  and  amongst  a  four  ship,  then  again,  that’s  where,  we’re 
going  into  something  that  kind  of  was  new  to  us  and  we 
have  to  be  very  flexible  in  a  very  rapid  amount  of  time  and 
that’s  again  when  you're  starting  to  think  about  the  fratricide. 
A  good  com  would  say,  “Viper  one,  group  four  ship  off  the 
container...  “  You’re  all  on  the  same  sheet  of  music  and 
now,  as  you  go  into  this  merge,  probably  good  things  will 
happen.  But  conversely,  let’s  say  that  the  com  is  terrible. 
There  is  starting  to  be  a  break  down,  our  situation 
awareness  isn't  as  high  as  you  go  into  this  merge.  And  if,  all 
of  a  sudden,  there's  a  com  break  down,  like  a  call  is  not 
made,  like  you  really  killed  somebody  or,  oh,  my  god,  there’s 
four  of  them  not  two  of  them,  then  all  of  a  sudden  again  your 
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Does  not  follow  game 
plan 


Does  not  follow  game 
plan 


Keeping  SA  via  comm 
going  into  a  merge 


Workload  during  a  merge  SA 


Identification, 

communication 


Communication 


chances  for  fratricide  has  gone  way  up. 

SA 

So  a  lot  of  it  is  listening  and  really  digesting  the  information. 
That’s  the  hardest  part  about  being  in  a  single  seat  airplane. 

SA 

Let’s  say  that  my  tactic  is  to  leave  at  range,  but  1  decide  I’m 
not  going  to  do  that,  I’m  going  to  go  closer  to  this  merge. 

Well,  now  1  may  have  red  airplanes  and  blue  airplanes 
mixed  up.  Let’s  say  that  was  number  three  and  four,  and 
when  one  and  two  start  looking  with  their  radars,  now  all  of  a 
sudden  they  have  a  chocolate  mess  to  deal  with,  versus 
where  if  the  guy  would  have  executed  the  game  plan  then  it 
would  have  been  a  nice  picture  to  deal  with. 

Now  1  may  not  be  able  to  employ  the  BVR,  because  now  1 
got  friendlies  and  hostiles  all  mixed  up  in  the  same  group, 
then  that’s  bad. 

SA 

Its  kind  of  assumed  that  if  a  guy  just  trundles  into  a  merge 
and  first  off,  he  could  die,  and  second  off  if  he  doesn't  die  its 
going  to  be  a  chocolate  mess.  And  that’s  pretty  much 
understood.  But  it  happens  because  guy  goes,  “Yeah,  1  can 
make  it  happen  1  can  do  it,  one  more  second  1  can  kill  these 
guys,”  and  it  doesn't  happen,  then  all  of  a  sudden  it’s  bad. 

SA 

1  think  the  hardest  arena  is  the  visual  identification  where 
you’ve  got  several  entities  that  you're  wading  into.  So,  let’s 
say  you’re  attacking  a  four  ship.  It  could  be  either  a  two  ship 
or  a  four  ship  of  blue  air,  but,  the  bottom  line  is  you  have  a 
completely  compressed  comm,  things  are  happening  so 
quick  because  you’re  closing,  every  three  seconds  you’re 
closing  a  mile.. .A  lot  of  things  are  happening,  so  your  comm 
has  to  be  pure  because  if  it’s  not,  then  you're  not  going  to  hit 
the  merge  with  the  most  situational  awareness. 

SA 

I’m  worried  about  where  my  wingman  is  and  I’m  worried 
about  what  numerous  people  are  doing  i.e.  like  you  know 
the  threats.  So  if  1  start  going  in  there  and  all  of  a  sudden 
they  start  breaking  left  and  right,  and  going  every  which 
way,  then  all  of  a  sudden  is  like. ..If  you  don’t  have 
identification  at  that  point,  it  really  gets  to  be  messy 

The  bottom  line  is  that  when  you  hit  that  merge  you  want 
to  be  able  to  make  somebody  go  away.  And  then,  hopefully 
maybe  even  two  people  go  away,  and  then  now,  if  you  took 
a  two  ship  in,  if  you  had  a  four,  if  you  were  attacking  a  four 
ship  in,  at  the  merge  you  made  two  people  go  away  almost 
instantaneously  now,  you  know  the  numbers  are  a  lot  easier 
to  deal  with. 

SA 

Take  a  look  at  an  F-18,  a  Fulcrum  and  a  F-16.  And 
sometimes  it  is  extremely  difficult  to  tell  who's  who. 

Well,  1  mean,  hopefully  you  shoot  the  right  person.  1  mean, 
that’s  when  com  comes  in,  you  may  go,  Viper  Two  status, 
and  he  may  say,  hey  I'm  a  little  guy  going  due  north.  Well  if 
you  see  two  guys  and  one  guy  is  high  and  one  guy  is  going 
due  south,  you  see  a  little  guy  going  due  north  you  may  sit 
there  and  go,  hey  man,  my  bro  is  in  trouble  and  you’re  going 
to  take,  kill  the  other  guy,  you  know,  but  that’s  based  off  of 
com. 

SA 

Typically,  BVR,  things  are  fairly  quiet  till  the  shots  are 
startinq  to  come.  When  does  it  qet,  you  know,  it  qets 

62 


difficult  maybe  when  the  groups  start  maneuvering  and  then 
the  comm  time  will  start  decreasing  because  there’ll  be  a  lot 
of  people  jabbering  about  what  the  guys  are  doing.  Hey,  the 
north  group  is  maneuvering,  flanking  south,  you  know,  so  at 
that  point  there  is  some  com  that  will  start  getting 

_ compressed. _ 

High  workload  situations  SA  I  think  the  hardest  the  hardest  thing  that  we  all  do  is  go  to  a 

visual  merge  against  multiple  enemies.  That  is  the  highest 
stress  level.  I  would  say  the  second  highest  stress  level  is 
when  we're  running  away  and  now  we  have  to  build  this 
mental  picture  of  what’s  happening  behind  us,  and  guys  did 
not  blow  up  like  they're  supposed  to  blow  up.  That's  a  fairly 
high  stress  level  because  your  SA  is  totally  based  on  com, 
now  between  you  and  AWACS  and  your  radar  works  is 
going  to  have  to  be  very  good,  particularly  if  we  have  guys 
that  are  chasing  me  out.  And  maybe,  the  third  would  be 
when  I’m  taking  shots  at  range  and  now  the  groups  start 
maneuvering  and,  you  know,  they're  not  doing  simple 
maneuvers,  we’re  talking  the  exploding  cantaloupe.  That’s  a 
fairly  high  level  of  stress  area,  too. 

Now,  you  can  turn  around  with  a  little  bit  more  situational 
awareness  and  you  got  range  to  deal  with  it.  But  if  you  sit 
there  and  see  this  exploding  cantaloupe  going  on,  and  you 
just  keep  driving  at  it,  and  you  are  not  good  going  at 

_ everybody  in  that  whole  exploding  cantaloupe  targeted... 

Communication  SA  Does  take  some  experience  to  get  used  to  using  all  the 

right  terminology?  Yeah.  Particularly  in  a  high  stress 
situation. 

You  forget  it  then  and  have  to  revert  to  longer  phrases? 

We  call  it  Navy  talk,  no.  No,  you  know,  I  use  that 
affectionately.  No,  that’s  exactly  the  case.  Communication 
between  two  people  is  relatively  simple  when  you’re  sitting 
there,  because  you’ve  got  the  facial  expressions,  the  hand 
gestures  and  all  that  stuff.  But  now  when  you’re  flying  an 
airplane,  you  don’t  see  of  that  and  everything  that  you  gain 
of  situational  awareness  from  that  guy  is  pretty  much 
through  communication.  If  your  definition  is  different  than 
somebody  else’s  definition  then  obviously  that  causes 

_ problems. _ 

Radar  mechanics  T&F,  SA  Well,  interpreting  the  radar  in  where  do  I  need  to  flow  my 

flight.  To  put  myself  in  an  advantageous  situation.  We 
always  see  problems  with  groups  doing  that.  Usually  they 
can  target  correctly,  their  shot  doctrine  is  usually  pretty 
decent.  I  think  putting  their  flight,  though,  in  a  more 
advantageous  position  is  sometimes  a  little  bit  of  a  limiting 
factor,  so  they  start  off  from  a  position  of  disadvantage, 
whereas  if  they  would  have  flowed  their  flight  a  little  better 
then  they  would  have  been  in  a  position  of  advantage. 

I  mean,  the  radar  will  display,  you  know,  where  things  are 
in  raw  form  and  then  you’ve  got  to,  you  have  to  sit  there  and 

_ go,  okay,  now  what  do  I  do  with  that  information? _ 

Prioritization  SA  He  may  miss-prioritize.  When  you  are  so  worried  about 

getting  the  shots  off  the  jet  that  he  is  dragging  himself  into 
an  area  where  he  could  be  shot.  So,  I  would  say  that  would 
_ be  a  miss-prioritization  and  he's  not  using  the  information  on 
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his  radar  to  prioritize. 

Let’s  say  we  go  back  to  our  tactics  again,  or  my  game 
plan,  let’s  say  that  I  want  my  flight  to  never  go  inside  a 
certain  range.  This  guy  is,  you  know,  he’s  out  there  and  he 
doesn’t  want  to  fail  and  he  wants  to  get  those  shots  in  that 
group  and  he  presses  his  two  ship  into  that  threat.  Well  it 
was  a  miss-prioritization  because  he's  trying  to  get  the  shots 
off  but  he’s  screwing  up  the  game  plan,  you  know,  trying  to 
do  that. _ 

Execution  Strategies 


Stick  to  the  game  plan 

SA 

If  1  tell  a  tactic  to  my  flight,  1  want  them  to  do  that  tactic  and 
don't  go  off  the  game  plan.  It’s  kind  of  like,  you’re  the 
quarterback  and  you’re  going  to  tell  a  guy,  look,  1  want  you 
to  run  ten  yards  down  the  field  and  then  turn  to  the  outside, 
towards  the  sideline,  I’m  going  to  throw  you  the  ball.  So,  he 
runs  ten  yards  and  goes,  to  the  inside,  it’s  going  to  be  a 
messed  up  play.  Well,  the  same  thing  happens  if  there's  a 
breakdown  in  the  game  plan.  Someone's  not  executing 
what  they’re  supposed  to  do  then,  all  of  a  sudden,  my 
situational  awareness  goes  down  and  that’s  when  we 
potentially  have  a  blue-on  -blue  problem.  So,  the  guys  have 
to  execute  the  game  plan,  and  other  than  that,  really,  the 
biggest  thing  about  blue-on-blue  is  when  you  enter  a  merge, 
a  visual  merge... 

Use  wingmen 

L, 

1  always  try  to  give  my  wingman  the  easiest  thing,  and  1 
leave  all  the  hard  decisions  for  myself.  So  1  tell  them,  “look 
if  1  tell  you  to  target  a  group  that  group  is  yours,  you  own 
that  group,  you  tell  if  the  group  is  doing  anything.”  The 
wingmen  that  do  that,  you  know,  we  are  a  better  fighting 
team,  the  wingmen  that  don't  do  that,  then  your  SA  goes 
down. 

Anyway,  so  1  always  give  my  wingman  the  easiest  problem, 
but,  no  kidding,  he  has  to  stick  to  the  contract  that  we  brief 
to,  like,  hey,  if  that  guy  maneuvers  and  trashes  your  shot  or 
he  maneuvers,  1  need  to  know  about  it,  you  know? 

Comm 

DP&C,  SA 

Well,  1  mean,  hopefully  you  shoot  the  right  person.  1  mean, 
that’s  when  com  comes  in,  you  may  go,  “Viper  2,  status,” 
and  he  may  say,  “I’m  a  little  guy  going  due  north.”  Well  if 
you  see  two  guys  and  one  guy  is  high  and  one  guy  is  going 
due  south,  you  see  a  little  guy  going  due  north  you  may  sit 
there  and  go,  hey  man,  my  bro  is  in  trouble  and  you’re  going 
to  take,  kill  the  other  guy,  you  know,  but  that’s  based  off  of 
com. 

SA 

Good  communication  will  keep  you  out  of  a  problem  area, 
and  as  long  as  guys  are  disciplined  with  your  shots,  then 
you  should  be  good  to  go.  But  if  guys  aren't  disciplined  with 
their  shots  and  the  communication  is  poor,  that  that’s  what’s 
going  to  happen,  someone’s  going  to  get  shot  by  their  own 
guy. 

Comm  priorities 

SA 

That’s  when  you  have  com  priorities.  Again,  if  they  turn 
sideways  where  they  have  gone  away  from  you.  They've 
turned  away  and  they  are  starting  to  run,  com  goes  up,  com 
time  opens  up  a  little  bit  because  yeah  they've  maneuvered 
but  they  are  going  away,  they  can't  kill  us  now.  So  once  the 
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initial  word  happens,  like  Viper  two  north  group  maneuver 
south. 

But  if  you  can  get  groups  going  sideways  or  dragging 
away  from  you,  at  least  your  range  isn't  collapsing  toward 
that  group.  The  com  time  is  a  little  bit  more  available 
because  you  have  range  on  your  side  versus  if  something  is 
happening  and  you’re  closing  in  on  the  group  that’s,  that’s 

_ when  you  start  getting  maxed  out. _ 

Run  away  for  exploding  DP&C,  T&F  If  I  see  an  exploding  cantaloupe,  I  try  to  run  away 

cantaloupe  because,  eventually,  if  they  want  to  kill  me  they’re  going 

have  to  come  back  towards  me. 

You’ll  talk  about  that,  a  lot  of  times  you’ll  sit  there  and  go, 
okay  if  groups  are  maneuvering,  we  are  just  going  to  turn 
around,  run  away  and  let  it  settle  out.  Because  if  they’re 
doing  exploding  cantaloupe,  they  can't  kill  you  and  they 
can't  kill  whatever  they  wanted  to  kill,  because  they  are 
doing  all  that  funky  maneuvering.  So  eventually  they'll  have 

_ to  come  back  hot,  and  settle  down... _ 

Maintain  contracts  SA,  MS  Is  there  anything  you  can  do  to  minimize  the  likelihood 

of  a  blue-on-blue  situation?  You  follow  the  ACM  rules. 
Who’s  the  engaged  fighter?  Who's  supporting  fighter?  Or, 
or  I  guess  probably  the  biggest  one  is,  who’s  the  engaged 
element?  Let’s  say  we  have  a  four  ship  again  and  we’re 
going  against  a  bogie  group  that  we  have  to  visually  identify. 
Well,  everybody  just  can’t  be  jumping  in  there  and  trying  to 
shoot  people,  because  what  happens  is,  if  you  don't  have 
very  good  com,  all  of  a  sudden,  you  might  be  getting  shot  by 
one  of  your  own  guys.  So,  that’s  why  it  is  very  important  for 
people  to  uphold  the  contracts  of  who’s  engaged  and  who's 

_ supporting. _ 

Preventing  blue  on  blue  SA  There’s  three  kinds  of  situational  awareness.  Positional 

situational  awareness  of  guys  in  my  flight.  One  is,  I  see 
them  on  my  radar.  Two  is  that,  through  communication, 
they  tell  me  where  they  are,  and  I  am  able  to  correlate  that 
and  go,  yep,  that’s  where  they  are,  okay?  Or,  three,  there 
are  some  data  link  systems  that  will  put  a  visual  picture  in 
your  aircraft  of  where  the  other  guys  are. 

They’re  all  a  combination.  It’s  nice  when  I  have  my,  if  all  of 
our,  you  guys  know  what  bull’s  eye  is? 

Okay.  I’m  talking  from  the  same  known  point,  so  let’s  call 
that  bull’s  eye,  all  right?  Then  if  my  number  three  man,  let’s 
say  I  don't  see  him,  I  don't  visually  look  out  my  window  and 
see  them  because  maybe  he’s  split  off,  but  he  goes,  I  am 
bull’s  eye  XYZ,  okay,  so  he  says  his  position  off  of  bull’s 
eye,  well  if  I  can  correlate  that  on  my  radar  or  I  can  correlate 
that  on  something  else,  then  that  is  great,  because  now  I 
have  two  sources  and  it  makes  my  job  easier.  I  feel  a  little 
better  about  it.  If  you’re  saying  he’s  somewhere,  and  I  can’t 
correlate  it,  then  it  kind  of  lowers  my  SA  a  little  bit,  I  mean,  I 
would  like  to  be  able  to  correlate  it  from  a  couple  different 
sources. 

Particularly  if  one  isn't  operating  correctly,  whether  it  be 
radar  or  my  data  link,  or  we’re  not,  we’re  getting  com 

_ jammed  and  I'm  not  getting  a  lot  of  com. _ 

2-ship  vs  4-ship  DP&C  Let’s  say  you  have  a  four  ship  that’s  supposed  to  be  on 
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station  for  two  hours  and  you  are  going  to  start  running  out 
of  gas,  so  what  happens  is  you’re  cycling  in  two  ship,  your 
wingman  is  going  to  be,  a  primary  targetter,  but,  if  I  have  two 
wing,  if  I  have  the  whole  force  up  there,  then  I  may  opt  to  do 

_ things  a  little  different. _ 

Use  of  wingman's  radar  DP&C,  SA  We're  both  out  there  sweeping,  and  now  we  need  to 

detect  something.  If  we  have  AWACS  or  ground  patrol 
intercept  radar,  it  make  it  a  little  bit  nicer  is,  assuming  they 
have  a  good  picture  of  all  the  operating  areas,  there  are  no 
shadows,  there’re  not  a  lot  of  mountains  where  maybe  bad 
guys  could  hide  behind,  or  whatever.  But  the  bottom  line  is 
we’ll  get  a  little  bit  of  information  from  them  and  using  that 
information  and  our  radar,  now  we  can  target  the 
appropriate  person.  I  mean,  like  I  said  before,  I’d  rather  use 
my  wingman’s  radar  at  range  to  target  in  there.  Why? 
Because  as  soon  as  you  take  a  radar  lock,  you’re  situational 
awareness  goes  down  a  little  bit  because  of  other  radar 
issues. 

Bottom  line  is,  you  want  to  use  his  radar  to  start  shooting 
people,  or  get  more  high  fidelity  information  and  then  you 
should  be  sweeping,  you  know,  so  you  can  have  a  big 
picture  of  what’s  going  on.  The  flight  leader’s  trying  to  keep 

_ the  big  picture. _ 

Use  of  commit  criteria  DP&C  You  want  to  engage  them  far  enough  away  so,  if  for 

some  reason  they  don’t  blow  up,  then  you  still  have  some 
time  to  take  care  of  them.  So  you  want  to  be  able  to  get 
them  right  when  they’re  meet  commit  criteria,  you  don't  want 
them  to  wait  back  too  far  where  you  have  to  start  really 
worrying  about,  you  know,  if  I  don't  this  right  the  first  time, 
we’re  screwed. 

The  bottom  line  is,  regardless  of  what  it  is,  whether  it’s 
hostile,  spades  or  bogey,  you're  going  to  commit  out  and 
then,  now,  knowing  what  they  are  helps  me  because  it’ll 
help  my  intercept  geometry.  Do  I  go  low?  Do  I  go  high? 

_ Things  like  that. _ 

Position  CAP  SA  If  you  have  a  couple  people  in  a  rectangular  room  and  you 

know,  you  had  people  lined  up  along  the,  you  know,  the  one 
end  of  the  room,  and  they  were  supposed  to  run  to  the  other 
end  of  the  room,  but  you  were  protecting  the  other  end  of 
the  room  and  you  had  to  tag  each  person,  you  probably 
wouldn't  want  to  be  against  one  wall,  you  know,  you 
probably  want  to  be  in  the  middle  so  you  can  go  half  way 
with  each,  right?  That’s  the  same  thing  for  an  airplane,  is  I 
have  to  position  my  cap  in  a  place  where  I  can  cover  my 
_ entire  area  that  I’m  responsible  for. _ 

Knowledge  that  Supports  Execution 

Comms  indicate  SA  SA  Because  you're  doing  multitasks,  the  first  thing  that  drops 

level,  how  mission  is  out...  You  can  tell,  when  someone's  SA  is  going  down,  com 

going  starts  going  to  shit.  Do  they  talk  too  much  or  not 

enough?  Well,  they’ll  do  both,  I  mean,  sometimes  they’ll 
say  nothing  when  they're  supposed  to  be  saying  something 
and  other  times  they’ll  be  spouting  off  about  stuff  that’s  not 

_ important  and  they’re  totally  miss-prioritizing. _ 

SA  Guys  know  that  if  com  is  just  ticking,  I’m  sorted  here,  yeah 

_ you’re  sorted  there,  oh  tally  four,  tally  four,  you  know,  as 
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you’re  coming  to  the  merge,  you  just  know  by  com,  things 
are  going  your  way.  But  if  you’re  not  hearing  that  com, 
that’s  when  you  sit  there  going,  uh-uh,  you  know,  things 

_ aren’t  going  my  way,  I  really  got  to  be  careful. _ 

SA  The  best  missions  are  the  ones  that  execute  the  game  plans 

and  have  very  good  com.  Not  too  much  com,  not  too  little 

_ com.  Com  when  when  it's  required. _ 

Visual  merge  SA,  L  But,  if  you  went  to  an  F-16  and  F-18  or  an  F-18  and  F-15  or 

Fulcrum,  then,  all  of  a  sudden  it  gets  a  just  a  little  more 
difficult. 

So,  whenever  there's  a  visual  merge,  the  hair  on  the  back  of 
my  cranium  stands  up  because  that’s  when  things  can 

_ happen. _ 

ACM  rules  SA  Well,  you  just,  everybody  just  can't  be  like,  jumping  in  there 

and  trying  to,  you  know,  shoot  people,  because  what 
happens,  always,  that  I’ve  seen,  well,  not  always  but,  most 
of  the  time  is,  if  you  don’t  have  very  good  com,  all  of  a 
sudden,  you  might  be  getting  shot  by  one  of  your  own  guys. 
So,  that’s  why  it  is  very  important  for  people,  again,  to 
uphold  the  contracts  of  who's  engaged  and  who’s 

_ supporting. _ 

Timeline  Awareness  and  T&F  But  if  you  wanted  to  break  down,  a  air  threat  coming  into  our 

Own  Capabilities  and  DCA  lane,  you’ve  got  your  cap  phase,  you've  got  your 

Limitations  (i.e.,  Missile  commit  phase,  you've  got  your  merge  preparation  phase 

Load,  Gas,  Radar)  and  your  merge  phase  and  post-merge 

The  commit,  we  talked  a  little  bit  about  already.  At  what 
point  am  I  going  to  sit  there  and  go,  I  need  to  leave,  excuse 
me,  I  need  to  leave  this  cap  and  go  and  engage  these 
people. 

The  merge  prep  is  nothing  more  than  getting  my 
appropriate  radars  down  there  to  target  the,  the  threats 
coming  my  way.  The  merge,  is  BVR  is  happening  at  range, 
so  a  merge  may  be  nothing  more  than,  now  we’re  starting  to 
employ  ordinance  on  these  guys  and  then  we’re  going  to, 
you  know,  clean  up  the  merge,  see  if  our  missiles  take  them 
out.  And  post-merge  is,  where  am  I  moving  my  forces  when 
I  come  off  this  engagement?  Where  am  I  flowing  to?  Am  I 
going  to  flow  down  to  the  south  to  take  care  of  somebody 
else  or  to  maybe  get  rejoined  with  my  flight?  That’s  what 

_ we’re  talking  about  in  post-merge. _ 

Knowledge  of  Element  DP&C  ...whenever  you  have  a  DCA  mission  anyway,  you’re  going 

(i.e.,  Load-Out,  Gas,  to  have  to  figure  out  what’s  your  criteria  for  your  commit. 

Capabilities  and  And  part  of  that’s  going  be  what  ordinance  you’re  carrying 

Limitations,  Roles  and  and  what  you’re  really  protecting.  Because  it  may  be  a  DCA 

Responsibilities)  lane  defense  mission  or  a  DCA  point  defense.  I  may  be 

given  a  block  that’s,  I  don’t  know,  say,  fifty  miles  wide  and  a 
hundred  miles  deep,  or  whatever... 

So,  first  you’ve  got  to  figure  out  the  kind  of  air  you’re 
playing  in  and  what  kind  of  ordinance  you’re  to  be  carrying, 
because  that’ll  have  some  effect  on  what  you’re  dong.  And 
also,  the  political  situation.  You  take  a  look  at  what's  out 
commit  criteria.  Is  it  only  when  they  come  across  the 
border?  Or,  if  they  commit  a  hostile  act  anywhere?  Is  that 
clearance  for  us  to  engage?  And  things  like  that.  So  that’s 
kind  of  first,  first,  I  guess,  decision  point,  is  okay,  what’s 
_ really  going  on  in  the  big  picture. _ 
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Load  out,  play  time 


DP&C,  T&F 


Wingman  responsibilities 


Contracts 


Mission  timeline  and 
decisions 


You’re  going  to  use  your  wingmen  and  pretty  much  get  their 
radars  into  the  groups.  And  if  they’re  declared  hostile,  then 
you’re  going  to  start  shooting  as  soon  as  you’re  in  a  web. 
When  you  shoot  depends  on  many  missiles  you  have  and 
how  long  you  have  to  be  on  station.  Obviously,  if  you  have 
to  be  on  station  for  five  hours,  your  shot  doctrine’s  going  to 
be  based  that.  But  the  bottom  line  is  that,  at  some  point, 
you’re  going  to  start  deploying  ordinance  if  you  have  a 
hostile  declaration,  and  you’re  going  to  do  that  for,  you 

know,  at  range. _ 

T&F,  MS  So,  to  answer  your  question,  it’s  pretty  much  standard,  I 

mean,  basically,  my  wingman’s  going  take  the  first  hostile, 
the  nearest  hostile  group  and  then,  after  that,  going  to  take 
what  we  call  spades  groups,  which  means  that,  you  know,  if 
we  know  that  a  group  isn't  squawking  the  right  kind  of 
codes,  we’re  going  to  see  if  we  can  update  them  to  a  hostile 
by  using  his  radar  and  lastly,  I’ll  put  my  wingman  on  bogie 
groups,  but  it  depends  on  if  you’re  using  a  grinder  or  well 

_ tactic... _ 

MS  Let’s  take  an  easy  problem.  There’re  just  two  of  us  out 

there.  And  my  contract  is  that  I  can  search  the  low  area,  so 
I’m  sweeping  and  I’m  looking  from  the  surface  up  to 
whatever  altitude,  and  then  my  wingman,  he’s  going  to  look 
with  a  little  bit  of  overlap  on  my  radar  coverage,  he  going  to 
look,  you  know,  say  fifteen  thousand  feet  and  up.  We  both 
have  a  contract  to  uphold,  because  if  I  look  high  and  he 

_ looks  high,  then  maybe  someone  can  sneak  in  down  low. 

DP&C,  T&F  Your  commit  criteria  may  be  something  like,  if  this  guy  is 

plus  or  minus  sixty  degrees  heading  of  0-9-0  so  if  he  is 
going  like  0-3-0  to  1-2-0  and  he's  crossing  that  line,  then  he 
may  be  committed  upon. 

That’s  kind  of  generic,  but  that  would  meet  what  we  call 
commit  criteria.  Now,  whether  he's  a  hostile,  spades  or 
bogie  is  going  to  influence  my  geometry  for  intercept.  Do  I 
go  high?  Do  I  go  low?  Do  I  offset  one  way  or  the  other?  So, 
your  commit  criteria  is  pretty  much  you’re  going  to  meet  that 
and  hopefully,  you're  going  to  have  declaration  so  it’ll 
optimize,  you  know,  your  geometry  to  get  these  guys. 

As  soon  as  we  know  what  is  coming  at  us,  like,  what  kind 
of  picture  is  it?  Are  they  groups  in  range?  Are  they  groups 
in  azimuth?  How  wide,  how  deep?  Then  we  can  figure  out 
our  game  plan.  If  I  have  a  four  ship,  I  may  not  want  to  send 
all  my  forces  forward  immediately,  I  may  want  to  send  a  two 
ship  to  kind  of  probe  a  little  bit  before  I  send  all  my  forces, 
so,  like  I  said,  now  the  tactic  part  comes  in,  is  okay,  these 
groups  are  meeting  commit  criteria, 

What,  what  is  really  coming  across  the  line?  Like  I  said, 
is  it  groups  in  range?  Groups  in  azimuth?  Is  it  a  bogie 
group  or  hostile  group,  and  I  choose  my  tactic  based  on 
what’s  coming  across  the  line. 

We  know  what  the  picture’s  coming  out  of,  so  you  have 
this  mental  picture,  let’s  just  say  it’s  two  group  range.  I  can 
commit  all  four  maybe,  okay,  so  we  all,  at  that  point  commit 
out,  we’re  starting  to  push  towards  the  threat.  Now,  I’m 
sitting  there  going,  okay,  two,  target  this  group.  So  he 
_ targets  it.  Now,  if  it's  a  hostile,  then  we  go  right  from  the 
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merge  prep  right  into  the  merge  phase,  where  now  we  start 
to  employ  ordinance. 

So,  as  long  as  they're  hostile,  you  go  right  to  the  employ 
ordinance  phase,  and  now,  he  tells  me  when  he  employs 
ordinance  and  I  maneuver  my  flight  to  a  better  position. 

So  right  there,  you've  gone  from  the  commit  to  the  merge 
prep,  we’re  now  throwing  radars  in  there... As  long  as  he's 
hostile,  yeah,  now  shots  will  come  off  the  rail. 

And  then  we  kind  of  go  into  post-merge  because  now  I’m 
maneuvering  my  flight  and  I  have  to  wait  and  see  how  my 
missiles  are  doing.  And  then  I  may  leave  or  may  go  clean 
up  the  merge. 

Whether  you  leave  or  clean  up--that’s  all  lumped  into  that 
post-merge  phase  because  you’re  making  a  decision  based 
on  what  just  happened,  okay?  Okay,  so  they’re  coming 
across  the  line,  there’re  four  of  them,  and  I  have  a  couple 
guys  taking  missile  shots  into  that  group,  okay?  Well,  how 
far  do  I  need  to  continue  to  go,  because  I’d  rather  not  go 
and,  when  I  say  merge,  I  mean,  like  no  kidding,  if  a  VID,  a 
visual  identification,  I  don’t  want  to  have  to  fly  right  by  them 
and  sit  there  and  go,  oh,  that  was  a  fulcrum.  I’d  rather  blow 
them  up  far  away.  Not  very  chivalrous,  but  that's  the  way 
war  is  now. 

If  I  get  the  shots  off,  now  I  have  to  decide,  do  I  want  to 
follow  the  shots  up?  I  mean,  no  kidding,  to  see  if  these  guys 
die,  or  do  I  just  want  to  leave?  Let’s  say  I  have  AWACS, 
well,  if  I  turn  my  tail  and  run  away,  okay,  are  my  missiles  are 
killing  them,  maybe  or  maybe  not.  Maybe  a  missile  fin  came 
off  from  a  missile,  maybe  the  fuse  didn’t  work,  you  know, 
maybe  well,  you  know,  if  there’s  instead  of  two  guys,  there’s 
really  four  guys  I  didn’t  know  about  and  neither  did  anybody 
else...  So,  there's  a  lot  of  decisions  that  you  have  to,  like  you 
said,  you  have  to  make  in  regards  to,  okay,  do  I  feel  like  I 
need  to  go  and  watch  this  whole  thing  explode,  or  can  I  just 

_ leave  and  pick  it  up  later? _ 

Knowledge  of  Distributed  DP&C,  T&F,  Let’s  talk  about  the  AWACS.  Let’s  say  I  have  a  four  ship.  If 

Teams  Capabilities  (i.e.,  SA  we’re  all  going  the  same  way,  then  eventually  in  my  CAP  I’m 

GCI,  F-18s,  F-5s)  going  to  have  to  turn  around  and  run  away,  and  go  away 

from  them  for  a  while.  If  we  don’t  have  AWACS,  we  may  not 
see  something  that  pops  up  on  the  scope.  So  if  I  know  that  I 
don't  have  AWACS,  I  may  have  to  adjust  the  kind  of 
formation  that  I’m  holding  my  CAP  in.  So  instead  of  maybe 
being  all  four  of  us  going  down  track  the  same  and  then 
turning  cold  and  running  away  then  turning  hot  and  running, 
you  know,  I  may  put  two  guys  going  cold  while  the  other  two 
guys  are  going  hot.  Now  my  radars  are  out  there  looking,  so 

_ I  always  have  somebody  looking  down  track. _ 

2-ship  versus  4-ship  DP&C,  T&F,  Well,  it  goes  back  to  my  tactic  of,  do  I  want  to  employ  us  as 

tactics  SA  an  entire  four  ship  or  not.  If  there's,  I  may  not  be  able  to 

react  quite  as  fast  if  I'm  broken  down  into  two  ships  and  I 
want  to  come  out  as  a  four  ship.  It  shouldn’t  be  that  much  of 
a  limiting  factor  though,  if  two  ship  is  running  hot  they  see 
something,  they  can  always  call  it  out,  great,  so  everybody, 
even  though  my  radar  is  not  on  them  and  I’m  going  cold,  I 
_ know  about  them,  and  then  what  I  could  do  is  sit  there  and 
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go,  okay,  well,  now,  I'm  going  to  turn  back  hot  and  hey 
number  three  I  want  you  to  rejoin  on  me  now,  so  we  can  go 
up  as  a  four  ship.  So,  it’s  not  that  big  of  a  deal  but  we  can 
get  at  least,  you  know,  radars  down,  some  radars  down 
there  and  then  we  can  come  up  with  a  game  plan.  The 
biggest  thing  about  no  AWACS  though,  at  least  for  F-16’s  is 
that  the  identification  of  the  bad  guys,  that’s  going  to  be  a 

_ limiting  factor,  and  I  won’t  go  more  into  that. _ 

AWACS  SA,  MS  Number  ten  is  what  ways  are  you  supported  by  the 

AWACS?  I  mean,  obviously  they  are  a  great  part  of  the 
team  because  they  are  basically  like  a  super  wingman.  The 
only  thing  is  they  can’t  kill  things  so,  that’s  why,  you  know, 
they  need  us. 

But  AWACS  will  be  able  to  help  us  out  by  maybe 
breaking  out  how  may  people  are  really  coming  across,  and 
what  kind  of  people  are  they,  and  are  they  hostiles,  spades 
or  boogie,  because  again  we  can’t  employ  ordinance  unless 
they're  hostile.  Plus,  because  they  are  such  a  big  airplane, 
they  can  see  further  than  the  radars  that  we  have,  so  they 
can  give  situational  awareness  ahead  of  what  our  radars 
would  tell  us.  It’s  kind  of  like  if  you  could  see  long  range 
driving  down  the  freeway  and  you  see  someone  swerving  in 
your  lane,  you  can  get  out  of  the  way,  right?  Well,  now  say 
you  put  a  blindfold  on  you  and  the  guy  swerves  into  you  lane 
and  then  now  all  of  a  sudden  take  it  off  of  you,  well,  your 
reaction  time  is  limited.  So  that’s  exactly  what  AWACS  does 
for  us,  they  give  that  long  range  look  that  we  can  set  up  our 
tactic  and  we  can  be  in  a  more  advantageous  position  to  kill 

_ them,  to  kill  the  bad  guys. _ 

AWACS  Commit  DP&C  Some  leads  will  allow  them  to  call  the  commit.  If  we’re 

authority  protecting  a  high  value  asset,  they  may  have  it  within  the 

rules  of  engagement  that  they  are  the  commit  authority,  so  it 
depends  on  how  the  rules  of  engagement  are,  but  typically 

_ fighters  have  the  commit  call,  AWACS  can  recommend  it. 

Enemy  Threat  DP&C,  ET  Let’s  say  that  I  have  to  protect  a  fifty  mile  wide  lane  for  three 

Capabilities  (i.e.,  SAs,  hours,  and  they’ve  been  doing  a  lot  of  probing  tactics,  like 

Intel  Brief)  they  come  flying  in,  but  then  they  turn  around  and  run.  Well, 

it’s  probably  better  for  me  not  to  go  just  wasting  my  missiles 
and  not  getting  anything  out  of  it.  If  they’re  defeating  all  of 
my  missiles  because  they  keep  running  away,  then,  if  I  don't 
know  that,  I’m  silly  and  if  I  keep  shooting  at  stuff  like  that 
that’s  silly,  because  pretty  soon,  I’ll  be  out  of,  out  of 

_ weapons  and  then  I’m  worthless. _ 

Shot  doctrine  T&F,  DP&C,  Remember  talking  about,  what  have  we  been  seeing  these 

ET  guys  do?  What  has  intel  told  us,  like,  maybe,  they’ve  been 

coming  out  in  two  groups  and  they've  been  maneuvering  at 
x  miles.  Well,  based  on  what  they're  doing,  based  on  my 
ordinance,  now  if  I  merge,  I'm  going  to  my  shot  doctrine. 
When  am  I  actually  going  to  shoot  this  missile?  Am  I  going 
to  shoot  it  way  far  away?  Am  I  going  to  shoot  it  closer? 
There’s,  I  can't  really  go  into  too  much  more  detail  than  that, 
but  the  bottom  line  is,  what  they’re  doing  and  what  you’re 
carrying,  and  how  many  missiles  you  have,  and  how  long  do 
you  have  to  be  there,  is  going  to  dictate  your  shot  doctrine. 
_ When  am  I  going  to  shoot  these  guys  for  the  first  time? 
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